[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:53:56 dist-test-slave-xb9n krb5kdc[31321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:53:56 dist-test-slave-xb9n krb5kdc[31321](info): set up 2 sockets
Jun 19 16:53:56 dist-test-slave-xb9n krb5kdc[31321](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 16:53:57.954270 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:53:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:53:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:53:58.083932 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:41969
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:41969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:53:58.162909 31334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:53:58.163118 31334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:53:58.163139 31334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:53:58.164251 31334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:53:58.164319 31334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:53:58.164340 31334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:53:58.164364 31334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:53:58.164389 31334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:53:58.165872 31334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:41969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:41969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:53:58.166098 31334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:53:58.166306 31334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:53:58.171002 31334 server_base.cc:1034] running on GCE node
W20260619 16:53:58.170987 31339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:53:58.170974 31342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:53:58.171025 31340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:53:58.171422 31334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:53:58.171696 31334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:53:58.172832 31334 hybrid_clock.cc:648] HybridClock initialized: now 1781888038172826 us; error 35 us; skew 500 ppm
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:53:58.175203 31334 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:53:58.175575 31334 webserver.cc:460] Webserver started at http://127.30.148.190:45163/ using document root <none> and password file <none>
I20260619 16:53:58.175758 31334 fs_manager.cc:359] Metadata directory not provided
I20260619 16:53:58.175797 31334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:53:58.175881 31334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:53:58.176813 31334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "f39ffdea71bc4a739df38f6b6a67b96c"
format_stamp: "Formatted at 2026-06-19 16:53:58 on dist-test-slave-xb9n"
server_key: "c91cf9660544898abbb933b92e7f05f5"
server_key_iv: "7dcd6098d6b5a37eaf798f1a45abc6e7"
server_key_version: "encryptionkey@0"
I20260619 16:53:58.177104 31334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "f39ffdea71bc4a739df38f6b6a67b96c"
format_stamp: "Formatted at 2026-06-19 16:53:58 on dist-test-slave-xb9n"
server_key: "c91cf9660544898abbb933b92e7f05f5"
server_key_iv: "7dcd6098d6b5a37eaf798f1a45abc6e7"
server_key_version: "encryptionkey@0"
I20260619 16:53:58.178691 31334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:53:58.179387 31349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:53:58.179559 31334 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:53:58.179638 31334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "f39ffdea71bc4a739df38f6b6a67b96c"
format_stamp: "Formatted at 2026-06-19 16:53:58 on dist-test-slave-xb9n"
server_key: "c91cf9660544898abbb933b92e7f05f5"
server_key_iv: "7dcd6098d6b5a37eaf798f1a45abc6e7"
server_key_version: "encryptionkey@0"
I20260619 16:53:58.179692 31334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:53:58.190292 31334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:53:58.192056 31334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:53:58.192165 31334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:53:58.195781 31334 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:41969
I20260619 16:53:58.195873 31401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:41969 every 8 connection(s)
I20260619 16:53:58.196149 31334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:53:58.196967 31402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:53:58.198483 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 31334
I20260619 16:53:58.198592 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:53:58.199396 31314 external_mini_cluster.cc:1428] Setting key e336d34c2f6ea3a09193199304552fdf
I20260619 16:53:58.199708 31402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c: Bootstrap starting.
I20260619 16:53:58.200372 31402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c: Neither blocks nor log segments found. Creating new log.
I20260619 16:53:58.200664 31402 log.cc:824] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c: Log is configured to *not* fsync() on all Append() calls
I20260619 16:53:58.201337 31402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c: No bootstrap required, opened a new log
I20260619 16:53:58.202986 31402 raft_consensus.cc:348] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39ffdea71bc4a739df38f6b6a67b96c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41969 } }
I20260619 16:53:58.203135 31402 raft_consensus.cc:374] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:53:58.203174 31402 raft_consensus.cc:729] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f39ffdea71bc4a739df38f6b6a67b96c, State: Initialized, Role: FOLLOWER
I20260619 16:53:58.203351 31402 consensus_queue.cc:260] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39ffdea71bc4a739df38f6b6a67b96c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41969 } }
I20260619 16:53:58.203445 31402 raft_consensus.cc:388] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:53:58.203476 31402 raft_consensus.cc:482] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:53:58.203532 31402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:53:58.204568 31402 raft_consensus.cc:504] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39ffdea71bc4a739df38f6b6a67b96c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41969 } }
I20260619 16:53:58.204684 31402 leader_election.cc:302] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f39ffdea71bc4a739df38f6b6a67b96c; no voters:
I20260619 16:53:58.204850 31402 leader_election.cc:288] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:53:58.204879 31407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:53:58.205068 31407 raft_consensus.cc:686] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [term 1 LEADER]: Becoming Leader. State: Replica: f39ffdea71bc4a739df38f6b6a67b96c, State: Running, Role: LEADER
I20260619 16:53:58.205116 31402 sys_catalog.cc:564] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:53:58.205176 31407 consensus_queue.cc:237] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39ffdea71bc4a739df38f6b6a67b96c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41969 } }
I20260619 16:53:58.205545 31407 sys_catalog.cc:455] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f39ffdea71bc4a739df38f6b6a67b96c. Latest consensus state: current_term: 1 leader_uuid: "f39ffdea71bc4a739df38f6b6a67b96c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39ffdea71bc4a739df38f6b6a67b96c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41969 } } }
I20260619 16:53:58.205621 31407 sys_catalog.cc:458] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [sys.catalog]: This master's current role is: LEADER
I20260619 16:53:58.205765 31408 sys_catalog.cc:455] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f39ffdea71bc4a739df38f6b6a67b96c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f39ffdea71bc4a739df38f6b6a67b96c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41969 } } }
I20260619 16:53:58.205828 31408 sys_catalog.cc:458] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c [sys.catalog]: This master's current role is: LEADER
I20260619 16:53:58.206149 31413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:53:58.206594 31413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:53:58.208132 31413 catalog_manager.cc:1347] Generated new cluster ID: e5efee29653f4defa94c7e6ef0ded651
I20260619 16:53:58.208185 31413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:53:58.217080 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:53:58.200346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59102 (local address 127.30.148.190:41969)
0619 16:53:58.200559 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 16:53:58.200568 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:53:58.200826 (+ 258us) server_negotiation.cc:407] Connection header received
0619 16:53:58.203022 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:53:58.203029 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:53:58.203080 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:53:58.203242 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:53:58.205468 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.206918 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.208279 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.208872 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.208958 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:53:58.212961 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:53:58.212964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:53:58.212964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:53:58.212974 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:53:58.214884 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:58.215323 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:58.215325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:58.215326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:58.215416 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:58.215645 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:58.215646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:58.215646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:58.215912 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:53:58.216048 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 16:53:58.216733 (+ 685us) server_negotiation.cc:299] Negotiation successful
0619 16:53:58.216848 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":79,"threads_started":1}
I20260619 16:53:58.222919 31413 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:53:58.223405 31413 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:53:58.228849 31413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f39ffdea71bc4a739df38f6b6a67b96c: Generated new TSK 0
I20260619 16:53:58.229018 31413 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:53:58.256233 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41969
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:53:58.334059 31428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:53:58.334263 31428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:53:58.334298 31428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:53:58.335362 31428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:53:58.335446 31428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:53:58.335543 31428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:53:58.336958 31428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41969
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:53:58.337208 31428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:53:58.337473 31428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:53:58.342108 31433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:53:58.342147 31434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:53:58.342278 31428 server_base.cc:1034] running on GCE node
W20260619 16:53:58.342108 31436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:53:58.342555 31428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:53:58.342799 31428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:53:58.343968 31428 hybrid_clock.cc:648] HybridClock initialized: now 1781888038343941 us; error 41 us; skew 500 ppm
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:53:58.346884 31428 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:53:58.347299 31428 webserver.cc:460] Webserver started at http://127.30.148.129:40273/ using document root <none> and password file <none>
I20260619 16:53:58.347525 31428 fs_manager.cc:359] Metadata directory not provided
I20260619 16:53:58.347595 31428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:53:58.347749 31428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:53:58.348690 31428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "e77d3c3a793c40a9a5cb0fae778a9f4f"
format_stamp: "Formatted at 2026-06-19 16:53:58 on dist-test-slave-xb9n"
server_key: "47b647f44d91c28a7f4bb3f535e3b66d"
server_key_iv: "194bf6a88f3000e836f4251cfcf588d7"
server_key_version: "encryptionkey@0"
I20260619 16:53:58.349035 31428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "e77d3c3a793c40a9a5cb0fae778a9f4f"
format_stamp: "Formatted at 2026-06-19 16:53:58 on dist-test-slave-xb9n"
server_key: "47b647f44d91c28a7f4bb3f535e3b66d"
server_key_iv: "194bf6a88f3000e836f4251cfcf588d7"
server_key_version: "encryptionkey@0"
I20260619 16:53:58.350374 31428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:53:58.351064 31443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:53:58.351226 31428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:53:58.351327 31428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "e77d3c3a793c40a9a5cb0fae778a9f4f"
format_stamp: "Formatted at 2026-06-19 16:53:58 on dist-test-slave-xb9n"
server_key: "47b647f44d91c28a7f4bb3f535e3b66d"
server_key_iv: "194bf6a88f3000e836f4251cfcf588d7"
server_key_version: "encryptionkey@0"
I20260619 16:53:58.351400 31428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:53:58.363233 31428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:53:58.364866 31428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:53:58.365020 31428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:53:58.365206 31428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:53:58.365541 31428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:53:58.365592 31428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:53:58.365631 31428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:53:58.365658 31428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:53:58.371387 31428 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:40361
I20260619 16:53:58.371474 31556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:40361 every 8 connection(s)
I20260619 16:53:58.371747 31428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:53:58.380259 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:53:58.372441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:57711 (local address 127.30.148.190:41969)
0619 16:53:58.372515 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 16:53:58.372518 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:53:58.372701 (+ 183us) server_negotiation.cc:407] Connection header received
0619 16:53:58.373102 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:53:58.373103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:53:58.373113 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:53:58.373165 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:53:58.374099 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.374588 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.375536 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.375727 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.375779 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:53:58.377771 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:53:58.377772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:53:58.377772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:53:58.377779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:53:58.378943 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:58.379319 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:58.379320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:58.379320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:58.379375 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:58.379638 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:58.379639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:58.379639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:58.379743 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:53:58.379814 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 16:53:58.380069 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 16:53:58.380172 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 16:53:58.380424 31559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:53:58.372417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41969 (local address 127.30.148.129:57711)
0619 16:53:58.372589 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 16:53:58.372620 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 16:53:58.373025 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:53:58.373218 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:53:58.373221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:53:58.373846 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:53:58.374017 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:53:58.374021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.374666 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.374666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:53:58.375453 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:53:58.375455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.375797 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.375797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:53:58.375887 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:53:58.376292 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:53:58.376299 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:53:58.377675 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:53:58.379023 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:53:58.379030 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:53:58.379032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:53:58.379242 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:53:58.379441 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:53:58.379441 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:53:58.379442 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:53:58.379564 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:53:58.379805 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:53:58.379811 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:53:58.379940 (+ 129us) client_negotiation.cc:769] Sending connection context
0619 16:53:58.380045 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 16:53:58.380203 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260619 16:53:58.380578 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 31428
I20260619 16:53:58.380676 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:53:58.380834 31314 external_mini_cluster.cc:1428] Setting key 6d9c6dde67bbe8a0556199df1fc99c47
I20260619 16:53:58.380913 31557 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41969
I20260619 16:53:58.381013 31557 heartbeater.cc:464] Registering TS with master...
I20260619 16:53:58.381207 31557 heartbeater.cc:511] Master 127.30.148.190:41969 requested a full tablet report, sending...
I20260619 16:53:58.381697 31366 ts_manager.cc:194] Registered new tserver with Master: e77d3c3a793c40a9a5cb0fae778a9f4f (127.30.148.129:40361)
I20260619 16:53:58.382350 31366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:57711
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:53:58.403478 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41969
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:53:58.480237 31562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:53:58.480406 31562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:53:58.480428 31562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:53:58.481549 31562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:53:58.481586 31562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:53:58.481631 31562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:53:58.483208 31562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41969
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:53:58.483419 31562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:53:58.483644 31562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:53:58.488287 31570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:53:58.488312 31567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:53:58.488299 31568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:53:58.488543 31562 server_base.cc:1034] running on GCE node
I20260619 16:53:58.488819 31562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:53:58.489056 31562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:53:58.490208 31562 hybrid_clock.cc:648] HybridClock initialized: now 1781888038490203 us; error 35 us; skew 500 ppm
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:53:58.492561 31562 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:53:58.492956 31562 webserver.cc:460] Webserver started at http://127.30.148.130:34565/ using document root <none> and password file <none>
I20260619 16:53:58.493173 31562 fs_manager.cc:359] Metadata directory not provided
I20260619 16:53:58.493242 31562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:53:58.493432 31562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:53:58.494401 31562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "0b1e522fa21545c7ae54ec0512c0d192"
format_stamp: "Formatted at 2026-06-19 16:53:58 on dist-test-slave-xb9n"
server_key: "b706cacb2c9475fa8555de787f562852"
server_key_iv: "ef7f79b46678366678622c15ac822c95"
server_key_version: "encryptionkey@0"
I20260619 16:53:58.494758 31562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "0b1e522fa21545c7ae54ec0512c0d192"
format_stamp: "Formatted at 2026-06-19 16:53:58 on dist-test-slave-xb9n"
server_key: "b706cacb2c9475fa8555de787f562852"
server_key_iv: "ef7f79b46678366678622c15ac822c95"
server_key_version: "encryptionkey@0"
I20260619 16:53:58.496016 31562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:53:58.496694 31577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:53:58.496887 31562 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:53:58.496977 31562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "0b1e522fa21545c7ae54ec0512c0d192"
format_stamp: "Formatted at 2026-06-19 16:53:58 on dist-test-slave-xb9n"
server_key: "b706cacb2c9475fa8555de787f562852"
server_key_iv: "ef7f79b46678366678622c15ac822c95"
server_key_version: "encryptionkey@0"
I20260619 16:53:58.497057 31562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:53:58.505405 31562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:53:58.507570 31562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:53:58.507722 31562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:53:58.507920 31562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:53:58.508198 31562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:53:58.508250 31562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:53:58.508280 31562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:53:58.508307 31562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:53:58.513710 31562 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:42435
I20260619 16:53:58.513777 31690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:42435 every 8 connection(s)
I20260619 16:53:58.514088 31562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:53:58.517194 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 31562
I20260619 16:53:58.517264 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:53:58.517460 31314 external_mini_cluster.cc:1428] Setting key 9d2ce0e106be5fd0af7ff452557c0278
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:53:58.524288 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:53:58.514731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:42959 (local address 127.30.148.190:41969)
0619 16:53:58.514775 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:53:58.514777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:53:58.514965 (+ 188us) server_negotiation.cc:407] Connection header received
0619 16:53:58.515356 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:53:58.515357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:53:58.515366 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:53:58.515412 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:53:58.516227 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.516704 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.517817 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.518114 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.518197 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:53:58.521479 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:53:58.521480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:53:58.521481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:53:58.521487 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:53:58.522918 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:58.523340 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:58.523341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:58.523341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:58.523397 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:58.523662 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:58.523663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:58.523663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:58.523790 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:53:58.523864 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:53:58.524120 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 16:53:58.524199 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:53:58.524482 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:53:58.514714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41969 (local address 127.30.148.130:42959)
0619 16:53:58.514848 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 16:53:58.514878 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 16:53:58.515277 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:53:58.515467 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:53:58.515469 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:53:58.515982 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:53:58.516150 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:53:58.516154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.516778 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.516779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:53:58.517719 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:53:58.517722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.519348 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.519349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:53:58.519453 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:53:58.519961 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:53:58.519969 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:53:58.521325 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:53:58.523029 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:53:58.523035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:53:58.523037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:53:58.523260 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:53:58.523481 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:53:58.523482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:53:58.523483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:53:58.523588 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:53:58.523856 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:53:58.523861 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:53:58.523992 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 16:53:58.524099 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 16:53:58.524256 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
I20260619 16:53:58.524936 31691 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41969
I20260619 16:53:58.525032 31691 heartbeater.cc:464] Registering TS with master...
I20260619 16:53:58.525216 31691 heartbeater.cc:511] Master 127.30.148.190:41969 requested a full tablet report, sending...
I20260619 16:53:58.525599 31366 ts_manager.cc:194] Registered new tserver with Master: 0b1e522fa21545c7ae54ec0512c0d192 (127.30.148.130:42435)
I20260619 16:53:58.525952 31366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:42959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:53:58.540442 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41969
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:53:58.617084 31696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:53:58.617261 31696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:53:58.617283 31696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:53:58.618420 31696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:53:58.618461 31696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:53:58.618487 31696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:53:58.619871 31696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41969
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:53:58.620035 31696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:53:58.620256 31696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:53:58.624898 31704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:53:58.624902 31701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:53:58.624912 31702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:53:58.625103 31696 server_base.cc:1034] running on GCE node
I20260619 16:53:58.625409 31696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:53:58.625669 31696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:53:58.626827 31696 hybrid_clock.cc:648] HybridClock initialized: now 1781888038626805 us; error 34 us; skew 500 ppm
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:53:58.629122 31696 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:53:58.629580 31696 webserver.cc:460] Webserver started at http://127.30.148.131:44365/ using document root <none> and password file <none>
I20260619 16:53:58.629809 31696 fs_manager.cc:359] Metadata directory not provided
I20260619 16:53:58.629875 31696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:53:58.630010 31696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:53:58.630965 31696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "15176bb41f134dfdab5963ad3f6e5915"
format_stamp: "Formatted at 2026-06-19 16:53:58 on dist-test-slave-xb9n"
server_key: "be3fdb42548366f7f53b2b282859da60"
server_key_iv: "a2b31f604c0e519da3bead65e0e343ac"
server_key_version: "encryptionkey@0"
I20260619 16:53:58.631349 31696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "15176bb41f134dfdab5963ad3f6e5915"
format_stamp: "Formatted at 2026-06-19 16:53:58 on dist-test-slave-xb9n"
server_key: "be3fdb42548366f7f53b2b282859da60"
server_key_iv: "a2b31f604c0e519da3bead65e0e343ac"
server_key_version: "encryptionkey@0"
I20260619 16:53:58.632584 31696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:53:58.633226 31711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:53:58.633424 31696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:53:58.633522 31696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "15176bb41f134dfdab5963ad3f6e5915"
format_stamp: "Formatted at 2026-06-19 16:53:58 on dist-test-slave-xb9n"
server_key: "be3fdb42548366f7f53b2b282859da60"
server_key_iv: "a2b31f604c0e519da3bead65e0e343ac"
server_key_version: "encryptionkey@0"
I20260619 16:53:58.633620 31696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:53:58.654505 31696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:53:58.656168 31696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:53:58.656322 31696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:53:58.656526 31696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:53:58.656824 31696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:53:58.656876 31696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:53:58.656914 31696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:53:58.656942 31696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:53:58.662560 31696 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:37199
I20260619 16:53:58.662593 31824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:37199 every 8 connection(s)
I20260619 16:53:58.662909 31696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:53:58.665731 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 31696
I20260619 16:53:58.665824 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:53:58.665980 31314 external_mini_cluster.cc:1428] Setting key 9415f1687ea94cdddf1101020273f04a
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:53:58.672348 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:53:58.663632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:37999 (local address 127.30.148.190:41969)
0619 16:53:58.663701 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 16:53:58.663703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:53:58.663944 (+ 241us) server_negotiation.cc:407] Connection header received
0619 16:53:58.664453 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:53:58.664454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:53:58.664465 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:53:58.664507 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:53:58.665568 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.666338 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.667800 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.668088 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.668165 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:53:58.670118 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:53:58.670119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:53:58.670119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:53:58.670125 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:53:58.671267 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:58.671610 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:58.671611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:58.671611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:58.671658 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:58.671866 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:58.671867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:58.671867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:58.671956 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:53:58.672003 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:53:58.672231 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 16:53:58.672285 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 16:53:58.672514 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:53:58.663609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41969 (local address 127.30.148.131:37999)
0619 16:53:58.663813 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 16:53:58.663862 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 16:53:58.664335 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:53:58.664571 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:53:58.664582 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:53:58.665145 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:53:58.665454 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:53:58.665459 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.666439 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.666440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:53:58.667696 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:53:58.667700 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.668168 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.668169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:53:58.668270 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:53:58.668678 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:53:58.668685 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:53:58.670037 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:53:58.671343 (+ 1306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:53:58.671349 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:53:58.671350 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:53:58.671548 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:53:58.671715 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:53:58.671716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:53:58.671716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:53:58.671812 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:53:58.672010 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:53:58.672013 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:53:58.672113 (+ 100us) client_negotiation.cc:769] Sending connection context
0619 16:53:58.672192 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 16:53:58.672301 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
I20260619 16:53:58.672964 31825 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41969
I20260619 16:53:58.673038 31825 heartbeater.cc:464] Registering TS with master...
I20260619 16:53:58.673204 31825 heartbeater.cc:511] Master 127.30.148.190:41969 requested a full tablet report, sending...
I20260619 16:53:58.673596 31366 ts_manager.cc:194] Registered new tserver with Master: 15176bb41f134dfdab5963ad3f6e5915 (127.30.148.131:37199)
I20260619 16:53:58.673949 31366 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:37999
I20260619 16:53:58.678335 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.148.129@KRBTEST.COM
I20260619 16:53:58.699710 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:53:58.692812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45618 (local address 127.30.148.129:40361)
0619 16:53:58.692973 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 16:53:58.692996 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0619 16:53:58.693005 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:53:58.693032 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:53:58.693033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:53:58.693045 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:53:58.693106 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:53:58.693697 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.694382 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.695082 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.695355 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.695403 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:53:58.697418 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:53:58.697424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:53:58.697425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:53:58.697430 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:53:58.698777 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:58.699070 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:58.699072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:58.699072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:58.699118 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:58.699275 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:58.699276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:58.699276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:58.699383 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:53:58.699489 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 16:53:58.699533 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 16:53:58.699642 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
W20260619 16:53:58.700052 31471 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45618
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.129@KRBTEST.COM
I20260619 16:53:58.720074 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:53:58.713783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45632 (local address 127.30.148.129:40361)
0619 16:53:58.713841 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 16:53:58.713843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:53:58.713853 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:53:58.713918 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:53:58.713918 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:53:58.713928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:53:58.713972 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:53:58.714466 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.714917 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.715587 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:58.715767 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:58.715813 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:53:58.717840 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:53:58.717842 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:53:58.717842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:53:58.717849 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:53:58.718974 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:58.719326 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:58.719327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:58.719328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:58.719379 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:58.719578 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:58.719580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:58.719580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:58.719683 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:53:58.719790 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 16:53:58.719879 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 16:53:58.719967 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 16:53:58.720585 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 31428
I20260619 16:53:58.725327 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 31562
I20260619 16:53:58.729419 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 31696
I20260619 16:53:58.733844 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 31334
2026-06-19T16:53:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1810 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31839](info): set up 2 sockets
Jun 19 16:53:58 dist-test-slave-xb9n krb5kdc[31839](info): commencing operation
krb5kdc: starting...
W20260619 16:53:59.752871 31314 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:53:59 dist-test-slave-xb9n krb5kdc[31839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:53:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:53:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:53:59.860376 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:38967
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:38967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:53:59.938181 31852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:53:59.938402 31852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:53:59.938439 31852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:53:59.939553 31852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:53:59.939622 31852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:53:59.939649 31852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:53:59.939680 31852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:53:59.939704 31852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:53:59.941143 31852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:38967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:38967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:53:59.941435 31852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:53:59.941641 31852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:53:59.946344 31852 server_base.cc:1034] running on GCE node
W20260619 16:53:59.946404 31857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:53:59.946369 31858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:53:59.946388 31860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:53:59.946746 31852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:53:59.947024 31852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:53:59.948167 31852 hybrid_clock.cc:648] HybridClock initialized: now 1781888039948152 us; error 27 us; skew 500 ppm
Jun 19 16:53:59 dist-test-slave-xb9n krb5kdc[31839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888039, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:53:59.950654 31852 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:53:59.951139 31852 webserver.cc:460] Webserver started at http://127.30.148.190:33831/ using document root <none> and password file <none>
I20260619 16:53:59.951372 31852 fs_manager.cc:359] Metadata directory not provided
I20260619 16:53:59.951452 31852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:53:59.951596 31852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:53:59.952642 31852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "f434fa89ee824178ba950d82f6b5211d"
format_stamp: "Formatted at 2026-06-19 16:53:59 on dist-test-slave-xb9n"
server_key: "3b2cc4cab5de427eac131814b7b3e384"
server_key_iv: "9fb39f02f59a9f43b1714deaffe44c6d"
server_key_version: "encryptionkey@0"
I20260619 16:53:59.953022 31852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "f434fa89ee824178ba950d82f6b5211d"
format_stamp: "Formatted at 2026-06-19 16:53:59 on dist-test-slave-xb9n"
server_key: "3b2cc4cab5de427eac131814b7b3e384"
server_key_iv: "9fb39f02f59a9f43b1714deaffe44c6d"
server_key_version: "encryptionkey@0"
I20260619 16:53:59.954438 31852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:53:59.955120 31867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:53:59.955281 31852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:53:59.955384 31852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "f434fa89ee824178ba950d82f6b5211d"
format_stamp: "Formatted at 2026-06-19 16:53:59 on dist-test-slave-xb9n"
server_key: "3b2cc4cab5de427eac131814b7b3e384"
server_key_iv: "9fb39f02f59a9f43b1714deaffe44c6d"
server_key_version: "encryptionkey@0"
I20260619 16:53:59.955452 31852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:53:59.967792 31852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:53:59.970012 31852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:53:59.970165 31852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:53:59.974110 31852 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:38967
I20260619 16:53:59.974139 31919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:38967 every 8 connection(s)
I20260619 16:53:59.974457 31852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:53:59.975234 31920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:53:59.977269 31920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d: Bootstrap starting.
I20260619 16:53:59.977815 31920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d: Neither blocks nor log segments found. Creating new log.
I20260619 16:53:59.978087 31920 log.cc:824] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d: Log is configured to *not* fsync() on all Append() calls
I20260619 16:53:59.978760 31920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d: No bootstrap required, opened a new log
I20260619 16:53:59.980182 31920 raft_consensus.cc:348] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f434fa89ee824178ba950d82f6b5211d" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38967 } }
I20260619 16:53:59.980300 31920 raft_consensus.cc:374] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:53:59.980321 31920 raft_consensus.cc:729] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f434fa89ee824178ba950d82f6b5211d, State: Initialized, Role: FOLLOWER
I20260619 16:53:59.980479 31920 consensus_queue.cc:260] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f434fa89ee824178ba950d82f6b5211d" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38967 } }
I20260619 16:53:59.980541 31920 raft_consensus.cc:388] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:53:59.980561 31920 raft_consensus.cc:482] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:53:59.980624 31920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:53:59.981091 31920 raft_consensus.cc:504] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f434fa89ee824178ba950d82f6b5211d" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38967 } }
I20260619 16:53:59.981190 31920 leader_election.cc:302] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f434fa89ee824178ba950d82f6b5211d; no voters:
I20260619 16:53:59.981388 31920 leader_election.cc:288] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:53:59.981438 31923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:53:59.981596 31923 raft_consensus.cc:686] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [term 1 LEADER]: Becoming Leader. State: Replica: f434fa89ee824178ba950d82f6b5211d, State: Running, Role: LEADER
I20260619 16:53:59.981627 31920 sys_catalog.cc:564] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:53:59.981683 31923 consensus_queue.cc:237] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f434fa89ee824178ba950d82f6b5211d" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38967 } }
I20260619 16:53:59.982060 31925 sys_catalog.cc:455] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f434fa89ee824178ba950d82f6b5211d. Latest consensus state: current_term: 1 leader_uuid: "f434fa89ee824178ba950d82f6b5211d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f434fa89ee824178ba950d82f6b5211d" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38967 } } }
I20260619 16:53:59.982147 31925 sys_catalog.cc:458] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [sys.catalog]: This master's current role is: LEADER
I20260619 16:53:59.982308 31924 sys_catalog.cc:455] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f434fa89ee824178ba950d82f6b5211d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f434fa89ee824178ba950d82f6b5211d" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38967 } } }
I20260619 16:53:59.982385 31932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:53:59.982417 31924 sys_catalog.cc:458] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d [sys.catalog]: This master's current role is: LEADER
I20260619 16:53:59.982913 31932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:53:59.984416 31932 catalog_manager.cc:1347] Generated new cluster ID: c5f905cedf2d4de7b93c13920da7f67d
I20260619 16:53:59.984468 31932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:53:59.985550 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 31852
I20260619 16:53:59.985661 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:53:59.985872 31314 external_mini_cluster.cc:1428] Setting key 1106eee09ff468548639323e9d99c9ae
Jun 19 16:53:59 dist-test-slave-xb9n krb5kdc[31839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:53:59.995208 31943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:53:59.986302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41418 (local address 127.30.148.190:38967)
0619 16:53:59.986458 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 16:53:59.986467 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:53:59.986547 (+ 80us) server_negotiation.cc:407] Connection header received
0619 16:53:59.986699 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:53:59.986704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:53:59.986753 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:53:59.986884 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:53:59.987465 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:59.988378 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:59.989137 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:53:59.989615 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:53:59.989744 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:53:59.992001 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:53:59.992005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:53:59.992006 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:53:59.992017 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:53:59.993696 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:59.994068 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:59.994071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:59.994071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:59.994156 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:53:59.994399 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:53:59.994400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:53:59.994401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:53:59.994630 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:53:59.994815 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 16:53:59.994880 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 16:53:59.995014 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":88,"threads_started":1}
I20260619 16:54:00.012419 31932 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:00.013010 31932 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:00.021616 31932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f434fa89ee824178ba950d82f6b5211d: Generated new TSK 0
I20260619 16:54:00.021854 31932 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:54:00.055737 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38967
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:00.136193 31946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:00.136386 31946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:00.136409 31946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:00.137573 31946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:00.137615 31946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:00.137666 31946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:00.139110 31946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38967
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:00.139376 31946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:00.139597 31946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:00.144534 31954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:00.144590 31952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:00.144550 31951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:00.144583 31946 server_base.cc:1034] running on GCE node
I20260619 16:54:00.144910 31946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:00.145172 31946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:00.146335 31946 hybrid_clock.cc:648] HybridClock initialized: now 1781888040146312 us; error 56 us; skew 500 ppm
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[31839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888040, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:00.148846 31946 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:54:00.149202 31946 webserver.cc:460] Webserver started at http://127.30.148.129:34773/ using document root <none> and password file <none>
I20260619 16:54:00.149449 31946 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:00.149497 31946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:00.149585 31946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:00.150537 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "e96d07979de64b9cbc90b94cdd423b3b"
format_stamp: "Formatted at 2026-06-19 16:54:00 on dist-test-slave-xb9n"
server_key: "2d4003aee714397cdaa69bac68bf30fd"
server_key_iv: "27ffada89404db24a73fe291cd7d1f84"
server_key_version: "encryptionkey@0"
I20260619 16:54:00.150828 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "e96d07979de64b9cbc90b94cdd423b3b"
format_stamp: "Formatted at 2026-06-19 16:54:00 on dist-test-slave-xb9n"
server_key: "2d4003aee714397cdaa69bac68bf30fd"
server_key_iv: "27ffada89404db24a73fe291cd7d1f84"
server_key_version: "encryptionkey@0"
I20260619 16:54:00.152042 31946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:54:00.152761 31961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:00.152979 31946 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:00.153074 31946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "e96d07979de64b9cbc90b94cdd423b3b"
format_stamp: "Formatted at 2026-06-19 16:54:00 on dist-test-slave-xb9n"
server_key: "2d4003aee714397cdaa69bac68bf30fd"
server_key_iv: "27ffada89404db24a73fe291cd7d1f84"
server_key_version: "encryptionkey@0"
I20260619 16:54:00.153182 31946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:00.173660 31946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:00.176064 31946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:00.176177 31946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:00.176373 31946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:00.176693 31946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:00.176729 31946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:00.176755 31946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:00.176806 31946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:00.182902 31946 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:42039
I20260619 16:54:00.182943 32074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:42039 every 8 connection(s)
I20260619 16:54:00.183382 31946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[31839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888040, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:00.190572 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 31946
I20260619 16:54:00.190694 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:00.190907 31314 external_mini_cluster.cc:1428] Setting key 076a2984cd3e1356f08cb18642951ad7
I20260619 16:54:00.192636 32077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.184058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:38967 (local address 127.30.148.129:45883)
0619 16:54:00.184227 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:00.184269 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:00.185176 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:00.185460 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:00.185463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:00.185717 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:00.185905 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.185909 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.186570 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.186571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.187374 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.187376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.187720 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.187721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.187819 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.188267 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:00.188275 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:00.189640 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:00.191304 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:00.191310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:00.191312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:00.191565 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:00.191784 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:00.191784 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:00.191784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:00.191885 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:00.192163 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:00.192167 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:00.192284 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 16:54:00.192366 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 16:54:00.192481 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260619 16:54:00.192674 31943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.184087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:45883 (local address 127.30.148.190:38967)
0619 16:54:00.184127 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:00.184129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:00.184361 (+ 232us) server_negotiation.cc:407] Connection header received
0619 16:54:00.185269 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:00.185269 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:00.185281 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:00.185336 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:00.185983 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.186484 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.187451 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.187648 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.187701 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.189726 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:00.189728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:00.189728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:00.189734 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:00.191219 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.191650 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.191651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.191651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.191718 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.191953 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.191954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.191954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.192097 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:00.192172 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:00.192408 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 16:54:00.192519 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:00.193418 32075 heartbeater.cc:347] Connected to a master server at 127.30.148.190:38967
I20260619 16:54:00.193528 32075 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:00.193722 32075 heartbeater.cc:511] Master 127.30.148.190:38967 requested a full tablet report, sending...
I20260619 16:54:00.194234 31884 ts_manager.cc:194] Registered new tserver with Master: e96d07979de64b9cbc90b94cdd423b3b (127.30.148.129:42039)
I20260619 16:54:00.195096 31884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:45883
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:54:00.214258 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38967
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:00.293589 32080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:00.293768 32080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:00.293789 32080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:00.294823 32080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:00.294853 32080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:00.294878 32080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:00.296269 32080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38967
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:00.296444 32080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:00.296613 32080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:00.301213 32088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:00.301226 32086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:00.301271 32080 server_base.cc:1034] running on GCE node
W20260619 16:54:00.301218 32085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:00.301589 32080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:00.301822 32080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:00.302974 32080 hybrid_clock.cc:648] HybridClock initialized: now 1781888040302943 us; error 44 us; skew 500 ppm
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[31839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888040, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:00.305266 32080 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:54:00.305725 32080 webserver.cc:460] Webserver started at http://127.30.148.130:37345/ using document root <none> and password file <none>
I20260619 16:54:00.305953 32080 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:00.306018 32080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:00.306176 32080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:00.307147 32080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "96ae48124f1a4f8a9c6e8515ffec895c"
format_stamp: "Formatted at 2026-06-19 16:54:00 on dist-test-slave-xb9n"
server_key: "cfe5d4be0deed1186ddbed0bcc52feb1"
server_key_iv: "a837827f8dba97e4a2fb9571f190b11c"
server_key_version: "encryptionkey@0"
I20260619 16:54:00.307509 32080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "96ae48124f1a4f8a9c6e8515ffec895c"
format_stamp: "Formatted at 2026-06-19 16:54:00 on dist-test-slave-xb9n"
server_key: "cfe5d4be0deed1186ddbed0bcc52feb1"
server_key_iv: "a837827f8dba97e4a2fb9571f190b11c"
server_key_version: "encryptionkey@0"
I20260619 16:54:00.308827 32080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:54:00.309525 32095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:00.309686 32080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:00.309777 32080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "96ae48124f1a4f8a9c6e8515ffec895c"
format_stamp: "Formatted at 2026-06-19 16:54:00 on dist-test-slave-xb9n"
server_key: "cfe5d4be0deed1186ddbed0bcc52feb1"
server_key_iv: "a837827f8dba97e4a2fb9571f190b11c"
server_key_version: "encryptionkey@0"
I20260619 16:54:00.309854 32080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:00.320576 32080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:00.322404 32080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:00.322563 32080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:00.322778 32080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:00.323083 32080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:00.323135 32080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:00.323175 32080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:00.323202 32080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:00.328959 32080 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:36419
I20260619 16:54:00.329005 32208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:36419 every 8 connection(s)
I20260619 16:54:00.329308 32080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[31839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888040, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:00.337883 31943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.330056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:50457 (local address 127.30.148.190:38967)
0619 16:54:00.330117 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:00.330120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:00.330318 (+ 198us) server_negotiation.cc:407] Connection header received
0619 16:54:00.331202 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:00.331203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:00.331212 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:00.331269 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:00.331821 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.332307 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.333233 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.333456 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.333501 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.335552 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:00.335553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:00.335553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:00.335559 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:00.336727 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.337073 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.337074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.337074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.337121 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.337327 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.337327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.337328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.337473 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:00.337526 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:00.337751 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 16:54:00.337823 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 16:54:00.338047 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.330023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:38967 (local address 127.30.148.130:50457)
0619 16:54:00.330195 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:00.330235 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:00.331118 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:00.331324 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:00.331327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:00.331563 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:00.331753 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.331758 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.332394 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.332395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.333163 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.333165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.333524 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.333525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.333620 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.334160 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:00.334176 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:00.335474 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:00.336798 (+ 1324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:00.336803 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:00.336804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:00.337010 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:00.337179 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:00.337179 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:00.337179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:00.337275 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:00.337530 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:00.337534 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:00.337642 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 16:54:00.337725 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 16:54:00.337857 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260619 16:54:00.338522 32209 heartbeater.cc:347] Connected to a master server at 127.30.148.190:38967
I20260619 16:54:00.338629 32209 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:00.338745 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 32080
I20260619 16:54:00.338827 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:00.338831 32209 heartbeater.cc:511] Master 127.30.148.190:38967 requested a full tablet report, sending...
I20260619 16:54:00.339010 31314 external_mini_cluster.cc:1428] Setting key e5cffe9427c4fb3247f1c721e678d49b
I20260619 16:54:00.339219 31884 ts_manager.cc:194] Registered new tserver with Master: 96ae48124f1a4f8a9c6e8515ffec895c (127.30.148.130:36419)
I20260619 16:54:00.339730 31884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:50457
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:54:00.362005 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38967
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:00.440678 32214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:00.440902 32214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:00.440939 32214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:00.442090 32214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:00.442157 32214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:00.442198 32214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:00.443703 32214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38967
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:00.443970 32214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:00.444197 32214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:00.448905 32219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:00.448982 32214 server_base.cc:1034] running on GCE node
W20260619 16:54:00.448889 32220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:00.448889 32222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:00.449323 32214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:00.449620 32214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:00.450771 32214 hybrid_clock.cc:648] HybridClock initialized: now 1781888040450754 us; error 30 us; skew 500 ppm
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[31839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888040, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:00.453182 32214 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:54:00.453646 32214 webserver.cc:460] Webserver started at http://127.30.148.131:37669/ using document root <none> and password file <none>
I20260619 16:54:00.453876 32214 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:00.453919 32214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:00.454051 32214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:00.455021 32214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "d360310c93964683846c9107ae7c66f7"
format_stamp: "Formatted at 2026-06-19 16:54:00 on dist-test-slave-xb9n"
server_key: "0103fa4859f9ef61528b8fee8f3508fb"
server_key_iv: "b5ee4bf4ad3676eea48a5b5b81a3d691"
server_key_version: "encryptionkey@0"
I20260619 16:54:00.455437 32214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "d360310c93964683846c9107ae7c66f7"
format_stamp: "Formatted at 2026-06-19 16:54:00 on dist-test-slave-xb9n"
server_key: "0103fa4859f9ef61528b8fee8f3508fb"
server_key_iv: "b5ee4bf4ad3676eea48a5b5b81a3d691"
server_key_version: "encryptionkey@0"
I20260619 16:54:00.456691 32214 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:00.457396 32229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:00.457564 32214 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:00.457670 32214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "d360310c93964683846c9107ae7c66f7"
format_stamp: "Formatted at 2026-06-19 16:54:00 on dist-test-slave-xb9n"
server_key: "0103fa4859f9ef61528b8fee8f3508fb"
server_key_iv: "b5ee4bf4ad3676eea48a5b5b81a3d691"
server_key_version: "encryptionkey@0"
I20260619 16:54:00.457751 32214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:00.477479 32214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:00.479631 32214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:00.479802 32214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:00.480024 32214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:00.480326 32214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:00.480378 32214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:00.480418 32214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:00.480448 32214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:00.486408 32214 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:39923
I20260619 16:54:00.486483 32342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:39923 every 8 connection(s)
I20260619 16:54:00.486794 32214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[31839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888040, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:00.495416 31943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.487494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:52053 (local address 127.30.148.190:38967)
0619 16:54:00.487574 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:00.487576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:00.487797 (+ 221us) server_negotiation.cc:407] Connection header received
0619 16:54:00.488697 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:00.488698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:00.488707 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:00.488765 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:00.489330 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.489831 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.490782 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.490953 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.491001 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.493064 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:00.493064 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:00.493065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:00.493069 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:00.494294 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.494638 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.494639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.494639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.494685 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.494886 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.494886 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.494886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.494975 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:00.495023 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:00.495303 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 16:54:00.495355 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 16:54:00.495718 32345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.487480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:38967 (local address 127.30.148.131:52053)
0619 16:54:00.487669 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:00.487709 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:00.488603 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:00.488819 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:00.488822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:00.489070 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:00.489259 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.489264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.489906 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.489907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.490709 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.490711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.491020 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.491021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.491117 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.491655 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:00.491663 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:00.492991 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:00.494362 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:00.494367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:00.494368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:00.494574 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:00.494741 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:00.494741 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:00.494742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:00.494833 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:00.495026 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:00.495029 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:00.495161 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 16:54:00.495288 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 16:54:00.495463 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260619 16:54:00.496210 32343 heartbeater.cc:347] Connected to a master server at 127.30.148.190:38967
I20260619 16:54:00.496323 32343 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:00.496433 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 32214
I20260619 16:54:00.496520 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:00.496524 32343 heartbeater.cc:511] Master 127.30.148.190:38967 requested a full tablet report, sending...
I20260619 16:54:00.496685 31314 external_mini_cluster.cc:1428] Setting key 2b29d06273d3c54b78a1a5c4a51f22d1
I20260619 16:54:00.496940 31884 ts_manager.cc:194] Registered new tserver with Master: d360310c93964683846c9107ae7c66f7 (127.30.148.131:39923)
I20260619 16:54:00.497397 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:54:00.497426 31884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:52053
I20260619 16:54:00.504180 31943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.499066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41422 (local address 127.30.148.190:38967)
0619 16:54:00.499101 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:00.499103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:00.499187 (+ 84us) server_negotiation.cc:407] Connection header received
0619 16:54:00.499237 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:00.499238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:00.499244 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:00.499290 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:00.499766 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.500213 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.500852 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.501014 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.501067 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.502175 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:00.502175 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:00.502176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:00.502180 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:00.503274 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.503583 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.503583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.503584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.503628 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.503782 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.503783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.503783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.503926 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:00.503996 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:00.504080 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 16:54:00.504128 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:00.506385 31884 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:00.507906 31884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:00.514184 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.510399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.30.148.130:36419)
0619 16:54:00.510557 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:00.510563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:00.510574 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:54:00.510617 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:00.510619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:00.510634 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:00.510716 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:00.510894 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.512036 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.513110 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.513806 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.513868 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.513903 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:00.514062 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 16:54:00.514093 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260619 16:54:00.514175 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.510146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:36419 (local address 127.0.0.1:56038)
0619 16:54:00.510266 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:00.510305 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:00.510371 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:00.510719 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:00.510723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:00.510725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:00.510810 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.510812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.512132 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.512133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.512985 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.512987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.513880 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.513881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.513955 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.513968 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 16:54:00.514044 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 16:54:00.514095 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":36,"thread_start_us":76,"threads_started":1}
I20260619 16:54:00.515008 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 864c05618d30471ba69762907bdcfa3a (DEFAULT_TABLE table=test-table [id=5889d71f1f6744ca95ede31dac38a505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:00.515123 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.510147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:39923 (local address 127.0.0.1:54636)
0619 16:54:00.510468 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:00.510476 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:00.510516 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:00.510747 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:00.510748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:00.510751 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:00.510827 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.510829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.512122 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.512123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.514063 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.514065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.514901 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.514902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.514964 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.514967 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:00.515022 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 16:54:00.515059 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":865,"thread_start_us":68,"threads_started":1}
I20260619 16:54:00.515133 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.510312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54636 (local address 127.30.148.131:39923)
0619 16:54:00.510592 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:00.510596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:00.510604 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:00.510632 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:00.510633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:00.510645 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:00.510733 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:00.510922 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.512004 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.514140 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.514784 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.514884 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.514950 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:00.515035 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 16:54:00.515068 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":85,"threads_started":1}
I20260619 16:54:00.515354 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 864c05618d30471ba69762907bdcfa3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:00.515760 32277 tablet_service.cc:1467] Processing CreateTablet for tablet 864c05618d30471ba69762907bdcfa3a (DEFAULT_TABLE table=test-table [id=5889d71f1f6744ca95ede31dac38a505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:00.516077 32277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 864c05618d30471ba69762907bdcfa3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:00.516911 32352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.510167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:42039 (local address 127.0.0.1:38640)
0619 16:54:00.510578 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:00.510587 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:00.510621 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:00.511020 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:00.511021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:00.511023 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:00.511068 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.511069 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.512104 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.512105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.515744 (+ 3639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.515747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.516667 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.516668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.516728 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.516731 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:00.516795 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 16:54:00.516834 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":1827,"thread_start_us":55,"threads_started":1}
I20260619 16:54:00.516929 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.510399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38640 (local address 127.30.148.129:42039)
0619 16:54:00.510861 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:00.510865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:00.510875 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:00.510910 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:00.510912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:00.510925 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:00.511022 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:00.511216 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.512002 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.515840 (+ 3838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.516563 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.516647 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.516696 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:00.516813 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 16:54:00.516854 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":77,"threads_started":1}
I20260619 16:54:00.518025 32009 tablet_service.cc:1467] Processing CreateTablet for tablet 864c05618d30471ba69762907bdcfa3a (DEFAULT_TABLE table=test-table [id=5889d71f1f6744ca95ede31dac38a505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:00.518352 32009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 864c05618d30471ba69762907bdcfa3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:00.518375 32357 tablet_bootstrap.cc:492] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c: Bootstrap starting.
I20260619 16:54:00.518983 32357 tablet_bootstrap.cc:654] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:00.519304 32358 tablet_bootstrap.cc:492] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7: Bootstrap starting.
I20260619 16:54:00.519582 32357 log.cc:824] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:00.519987 32358 tablet_bootstrap.cc:654] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:00.520442 32357 tablet_bootstrap.cc:492] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c: No bootstrap required, opened a new log
I20260619 16:54:00.520516 32357 ts_tablet_manager.cc:1397] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:00.520566 32358 log.cc:824] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:00.521351 32358 tablet_bootstrap.cc:492] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7: No bootstrap required, opened a new log
I20260619 16:54:00.521451 32358 ts_tablet_manager.cc:1397] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:54:00.521570 32361 tablet_bootstrap.cc:492] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b: Bootstrap starting.
I20260619 16:54:00.522198 32361 tablet_bootstrap.cc:654] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:00.522646 32361 log.cc:824] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:00.522639 32357 raft_consensus.cc:348] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96ae48124f1a4f8a9c6e8515ffec895c" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 36419 } } peers { permanent_uuid: "d360310c93964683846c9107ae7c66f7" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39923 } } peers { permanent_uuid: "e96d07979de64b9cbc90b94cdd423b3b" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42039 } }
I20260619 16:54:00.522819 32357 raft_consensus.cc:374] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:00.522861 32357 raft_consensus.cc:729] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ae48124f1a4f8a9c6e8515ffec895c, State: Initialized, Role: FOLLOWER
I20260619 16:54:00.522969 32357 consensus_queue.cc:260] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96ae48124f1a4f8a9c6e8515ffec895c" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 36419 } } peers { permanent_uuid: "d360310c93964683846c9107ae7c66f7" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39923 } } peers { permanent_uuid: "e96d07979de64b9cbc90b94cdd423b3b" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42039 } }
I20260619 16:54:00.523252 32357 ts_tablet_manager.cc:1428] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 16:54:00.523283 32209 heartbeater.cc:503] Master 127.30.148.190:38967 was elected leader, sending a full tablet report...
I20260619 16:54:00.523366 32358 raft_consensus.cc:348] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96ae48124f1a4f8a9c6e8515ffec895c" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 36419 } } peers { permanent_uuid: "d360310c93964683846c9107ae7c66f7" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39923 } } peers { permanent_uuid: "e96d07979de64b9cbc90b94cdd423b3b" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42039 } }
I20260619 16:54:00.523550 32358 raft_consensus.cc:374] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:00.523623 32358 raft_consensus.cc:729] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d360310c93964683846c9107ae7c66f7, State: Initialized, Role: FOLLOWER
I20260619 16:54:00.523738 32358 consensus_queue.cc:260] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96ae48124f1a4f8a9c6e8515ffec895c" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 36419 } } peers { permanent_uuid: "d360310c93964683846c9107ae7c66f7" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39923 } } peers { permanent_uuid: "e96d07979de64b9cbc90b94cdd423b3b" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42039 } }
I20260619 16:54:00.523895 32361 tablet_bootstrap.cc:492] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b: No bootstrap required, opened a new log
I20260619 16:54:00.523970 32361 ts_tablet_manager.cc:1397] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:00.524020 32358 ts_tablet_manager.cc:1428] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:54:00.524093 32343 heartbeater.cc:503] Master 127.30.148.190:38967 was elected leader, sending a full tablet report...
I20260619 16:54:00.525941 32361 raft_consensus.cc:348] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96ae48124f1a4f8a9c6e8515ffec895c" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 36419 } } peers { permanent_uuid: "d360310c93964683846c9107ae7c66f7" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39923 } } peers { permanent_uuid: "e96d07979de64b9cbc90b94cdd423b3b" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42039 } }
I20260619 16:54:00.526063 32361 raft_consensus.cc:374] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:00.526084 32361 raft_consensus.cc:729] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96d07979de64b9cbc90b94cdd423b3b, State: Initialized, Role: FOLLOWER
I20260619 16:54:00.526151 32361 consensus_queue.cc:260] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96ae48124f1a4f8a9c6e8515ffec895c" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 36419 } } peers { permanent_uuid: "d360310c93964683846c9107ae7c66f7" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39923 } } peers { permanent_uuid: "e96d07979de64b9cbc90b94cdd423b3b" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42039 } }
I20260619 16:54:00.526341 32361 ts_tablet_manager.cc:1428] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 16:54:00.526388 32075 heartbeater.cc:503] Master 127.30.148.190:38967 was elected leader, sending a full tablet report...
W20260619 16:54:00.579860 32210 tablet.cc:2369] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:54:00.684095 32076 tablet.cc:2369] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:54:00.686497 32364 raft_consensus.cc:482] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:00.686604 32364 raft_consensus.cc:504] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96ae48124f1a4f8a9c6e8515ffec895c" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 36419 } } peers { permanent_uuid: "d360310c93964683846c9107ae7c66f7" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39923 } } peers { permanent_uuid: "e96d07979de64b9cbc90b94cdd423b3b" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42039 } }
I20260619 16:54:00.686934 32364 leader_election.cc:288] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96ae48124f1a4f8a9c6e8515ffec895c (127.30.148.130:36419), e96d07979de64b9cbc90b94cdd423b3b (127.30.148.129:42039)
I20260619 16:54:00.689836 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.687200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:52651 (local address 127.30.148.129:42039)
0619 16:54:00.687264 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:00.687266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:00.687276 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:00.687342 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:00.687343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:00.687352 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:00.687428 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:00.687591 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.688064 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.689032 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.689474 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.689567 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.689575 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:00.689729 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 16:54:00.689765 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 16:54:00.689836 32345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.687149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:42039 (local address 127.30.148.131:52651)
0619 16:54:00.687193 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:00.687207 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:00.687271 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:00.687431 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:00.687432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:00.687434 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:00.687519 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.687520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.688148 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.688149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.688933 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.688935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.689565 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.689566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.689645 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.689648 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:00.689721 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 16:54:00.689767 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":28}
I20260619 16:54:00.689934 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.687248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:58945 (local address 127.30.148.130:36419)
0619 16:54:00.687306 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:00.687309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:00.687387 (+ 78us) server_negotiation.cc:407] Connection header received
0619 16:54:00.687459 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:00.687460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:00.687468 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:00.687569 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:00.687720 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.688178 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.689143 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.689570 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.689666 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.689675 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:00.689835 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 16:54:00.689871 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 16:54:00.689945 32366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.687150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:36419 (local address 127.30.148.131:58945)
0619 16:54:00.687306 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:00.687316 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:00.687404 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:00.687574 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:00.687574 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:00.687576 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:00.687649 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.687650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.688273 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.688274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.689036 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:00.689038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.689681 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.689682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:00.689751 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.689755 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:00.689827 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 16:54:00.689866 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":12,"thread_start_us":85,"threads_started":1}
I20260619 16:54:00.690308 32163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "864c05618d30471ba69762907bdcfa3a" candidate_uuid: "d360310c93964683846c9107ae7c66f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ae48124f1a4f8a9c6e8515ffec895c" is_pre_election: true
I20260619 16:54:00.690325 32029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "864c05618d30471ba69762907bdcfa3a" candidate_uuid: "d360310c93964683846c9107ae7c66f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96d07979de64b9cbc90b94cdd423b3b" is_pre_election: true
I20260619 16:54:00.690469 32029 raft_consensus.cc:2454] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d360310c93964683846c9107ae7c66f7 in term 0.
I20260619 16:54:00.690469 32163 raft_consensus.cc:2454] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d360310c93964683846c9107ae7c66f7 in term 0.
I20260619 16:54:00.690704 32233 leader_election.cc:302] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96ae48124f1a4f8a9c6e8515ffec895c, d360310c93964683846c9107ae7c66f7; no voters:
I20260619 16:54:00.690819 32364 raft_consensus.cc:2781] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:00.690871 32364 raft_consensus.cc:482] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:00.690904 32364 raft_consensus.cc:3037] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:00.691586 32364 raft_consensus.cc:504] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96ae48124f1a4f8a9c6e8515ffec895c" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 36419 } } peers { permanent_uuid: "d360310c93964683846c9107ae7c66f7" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39923 } } peers { permanent_uuid: "e96d07979de64b9cbc90b94cdd423b3b" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42039 } }
I20260619 16:54:00.691727 32364 leader_election.cc:288] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [CANDIDATE]: Term 1 election: Requested vote from peers 96ae48124f1a4f8a9c6e8515ffec895c (127.30.148.130:36419), e96d07979de64b9cbc90b94cdd423b3b (127.30.148.129:42039)
I20260619 16:54:00.691880 32163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "864c05618d30471ba69762907bdcfa3a" candidate_uuid: "d360310c93964683846c9107ae7c66f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ae48124f1a4f8a9c6e8515ffec895c"
I20260619 16:54:00.691887 32029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "864c05618d30471ba69762907bdcfa3a" candidate_uuid: "d360310c93964683846c9107ae7c66f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96d07979de64b9cbc90b94cdd423b3b"
I20260619 16:54:00.691974 32163 raft_consensus.cc:3037] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:00.691972 32029 raft_consensus.cc:3037] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:00.692660 32163 raft_consensus.cc:2454] T 864c05618d30471ba69762907bdcfa3a P 96ae48124f1a4f8a9c6e8515ffec895c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d360310c93964683846c9107ae7c66f7 in term 1.
I20260619 16:54:00.692672 32029 raft_consensus.cc:2454] T 864c05618d30471ba69762907bdcfa3a P e96d07979de64b9cbc90b94cdd423b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d360310c93964683846c9107ae7c66f7 in term 1.
I20260619 16:54:00.692848 32233 leader_election.cc:302] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96ae48124f1a4f8a9c6e8515ffec895c, d360310c93964683846c9107ae7c66f7; no voters:
I20260619 16:54:00.692934 32364 raft_consensus.cc:2781] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:00.693090 32364 raft_consensus.cc:686] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [term 1 LEADER]: Becoming Leader. State: Replica: d360310c93964683846c9107ae7c66f7, State: Running, Role: LEADER
I20260619 16:54:00.693261 32364 consensus_queue.cc:237] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96ae48124f1a4f8a9c6e8515ffec895c" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 36419 } } peers { permanent_uuid: "d360310c93964683846c9107ae7c66f7" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39923 } } peers { permanent_uuid: "e96d07979de64b9cbc90b94cdd423b3b" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42039 } }
I20260619 16:54:00.694089 31884 catalog_manager.cc:5495] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d360310c93964683846c9107ae7c66f7 (127.30.148.131). New cstate: current_term: 1 leader_uuid: "d360310c93964683846c9107ae7c66f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96ae48124f1a4f8a9c6e8515ffec895c" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 36419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d360310c93964683846c9107ae7c66f7" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e96d07979de64b9cbc90b94cdd423b3b" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42039 } health_report { overall_health: UNKNOWN } } }
W20260619 16:54:00.737435 32344 tablet.cc:2369] T 864c05618d30471ba69762907bdcfa3a P d360310c93964683846c9107ae7c66f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[31839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.129@KRBTEST.COM
I20260619 16:54:00.746788 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.739347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38642 (local address 127.30.148.129:42039)
0619 16:54:00.739421 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:00.739423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:00.739446 (+ 23us) server_negotiation.cc:407] Connection header received
0619 16:54:00.739520 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:00.739520 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:00.739547 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:00.739597 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:00.740143 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.740624 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.741458 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.741647 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.741702 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.744116 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:00.744127 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:00.744127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:00.744134 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:00.745847 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.746144 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.746147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.746147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.746206 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.746357 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.746357 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.746357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.746476 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:00.746589 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:00.746635 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 16:54:00.746693 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[31839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[31839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.148.129@KRBTEST.COM
I20260619 16:54:00.769534 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.762804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38654 (local address 127.30.148.129:42039)
0619 16:54:00.762887 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:00.762890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:00.762958 (+ 68us) server_negotiation.cc:407] Connection header received
0619 16:54:00.763068 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:00.763069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:00.763079 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:00.763151 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:00.763690 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.764164 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.764889 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.765086 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.765173 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.767330 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:00.767331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:00.767331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:00.767337 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:00.768555 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.768852 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.768853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.768853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.768900 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.769054 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.769054 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.769054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.769139 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:00.769207 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:00.769348 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 16:54:00.769468 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[31839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[31839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.129@KRBTEST.COM
I20260619 16:54:00.790783 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:00.783977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38670 (local address 127.30.148.129:42039)
0619 16:54:00.784045 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:00.784048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:00.784124 (+ 76us) server_negotiation.cc:407] Connection header received
0619 16:54:00.784229 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:00.784230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:00.784240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:00.784289 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:00.784809 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.785275 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.786075 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:00.786267 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:00.786353 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:00.788442 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:00.788444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:00.788444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:00.788452 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:00.789658 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.790034 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.790035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.790035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.790094 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:00.790292 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:00.790293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:00.790293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:00.790390 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:00.790494 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:00.790607 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 16:54:00.790675 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260619 16:54:00.791236 31989 tablet_service.cc:2908] Rejecting scan request for tablet 864c05618d30471ba69762907bdcfa3a: Uninitialized: clean time has not yet been initialized
I20260619 16:54:00.792083 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 31946
I20260619 16:54:00.796923 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 32080
I20260619 16:54:00.801205 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 32214
I20260619 16:54:00.805439 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 31852
2026-06-19T16:54:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2071 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[32381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[32381](info): set up 2 sockets
Jun 19 16:54:00 dist-test-slave-xb9n krb5kdc[32381](info): commencing operation
krb5kdc: starting...
W20260619 16:54:01.828805 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:01 dist-test-slave-xb9n krb5kdc[32381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:54:01.935750 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33707
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:33707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:02.014882 32394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:02.015116 32394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:02.015153 32394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:02.016278 32394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:02.016342 32394 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 16:54:02.016371 32394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:02.016402 32394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:02.016429 32394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:02.016460 32394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:02.017935 32394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:33707
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:02.018203 32394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:02.018450 32394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:02.023223 32399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:02.023329 32394 server_base.cc:1034] running on GCE node
W20260619 16:54:02.023254 32402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:02.023253 32400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:02.023775 32394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:02.024055 32394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:02.025195 32394 hybrid_clock.cc:648] HybridClock initialized: now 1781888042025173 us; error 47 us; skew 500 ppm
Jun 19 16:54:02 dist-test-slave-xb9n krb5kdc[32381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888042, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:02.030913 32394 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:54:02.031339 32394 webserver.cc:460] Webserver started at http://127.30.148.190:43959/ using document root <none> and password file <none>
I20260619 16:54:02.031560 32394 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:02.031628 32394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:02.031776 32394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:02.032819 32394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "e0bfa115380a4bb3a19bf88fde1850f0"
format_stamp: "Formatted at 2026-06-19 16:54:02 on dist-test-slave-xb9n"
server_key: "c28fcb5e60b9638534571facd634399e"
server_key_iv: "f751869d8d5f1e33c2cd92984ad670a1"
server_key_version: "encryptionkey@0"
I20260619 16:54:02.033190 32394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "e0bfa115380a4bb3a19bf88fde1850f0"
format_stamp: "Formatted at 2026-06-19 16:54:02 on dist-test-slave-xb9n"
server_key: "c28fcb5e60b9638534571facd634399e"
server_key_iv: "f751869d8d5f1e33c2cd92984ad670a1"
server_key_version: "encryptionkey@0"
I20260619 16:54:02.034520 32394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:02.035218 32409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.035390 32394 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:02.035483 32394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "e0bfa115380a4bb3a19bf88fde1850f0"
format_stamp: "Formatted at 2026-06-19 16:54:02 on dist-test-slave-xb9n"
server_key: "c28fcb5e60b9638534571facd634399e"
server_key_iv: "f751869d8d5f1e33c2cd92984ad670a1"
server_key_version: "encryptionkey@0"
I20260619 16:54:02.035568 32394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:02.055107 32394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:02.056869 32394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:02.056980 32394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:02.061193 32394 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:33707
I20260619 16:54:02.061311 32471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:33707 every 8 connection(s)
I20260619 16:54:02.061630 32394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:02.062283 32472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:02.064409 32472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0: Bootstrap starting.
I20260619 16:54:02.064929 32472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:02.065153 32472 log.cc:824] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:02.065830 32472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0: No bootstrap required, opened a new log
I20260619 16:54:02.067811 32472 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0bfa115380a4bb3a19bf88fde1850f0" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33707 } }
I20260619 16:54:02.067942 32472 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:02.067963 32472 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0bfa115380a4bb3a19bf88fde1850f0, State: Initialized, Role: FOLLOWER
I20260619 16:54:02.068059 32472 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0bfa115380a4bb3a19bf88fde1850f0" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33707 } }
I20260619 16:54:02.068138 32472 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:02.068171 32472 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:02.068229 32472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:02.068732 32472 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0bfa115380a4bb3a19bf88fde1850f0" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33707 } }
I20260619 16:54:02.068858 32472 leader_election.cc:302] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0bfa115380a4bb3a19bf88fde1850f0; no voters:
I20260619 16:54:02.069028 32472 leader_election.cc:288] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:02.069083 32475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:02.069255 32475 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [term 1 LEADER]: Becoming Leader. State: Replica: e0bfa115380a4bb3a19bf88fde1850f0, State: Running, Role: LEADER
I20260619 16:54:02.069293 32472 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:02.069411 32475 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0bfa115380a4bb3a19bf88fde1850f0" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33707 } }
I20260619 16:54:02.069780 32476 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0bfa115380a4bb3a19bf88fde1850f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0bfa115380a4bb3a19bf88fde1850f0" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33707 } } }
I20260619 16:54:02.069921 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 32394
I20260619 16:54:02.069936 32476 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:02.070015 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:02.070192 31314 external_mini_cluster.cc:1428] Setting key e8a5e1744a9349af1e7d3586fc1e13b4
I20260619 16:54:02.070684 32477 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0bfa115380a4bb3a19bf88fde1850f0. Latest consensus state: current_term: 1 leader_uuid: "e0bfa115380a4bb3a19bf88fde1850f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0bfa115380a4bb3a19bf88fde1850f0" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33707 } } }
I20260619 16:54:02.070950 32477 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0 [sys.catalog]: This master's current role is: LEADER
W20260619 16:54:02.071293 32492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:54:02.071350 32492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:54:02.071403 32483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:02.071607 32483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:02.073091 32483 catalog_manager.cc:1347] Generated new cluster ID: c41b0027dad54ebd9b501b771d8e5ede
I20260619 16:54:02.073143 32483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:54:02 dist-test-slave-xb9n krb5kdc[32381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:02.082695 32483 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:02.082967 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.070675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.30.148.190:33707)
0619 16:54:02.070965 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.070975 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.071008 (+ 33us) server_negotiation.cc:407] Connection header received
0619 16:54:02.071136 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.071142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.071182 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.071323 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:02.072087 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.073443 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.074204 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.074627 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.074744 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.079597 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:02.079606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:02.079607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:02.079618 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:02.081299 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.081743 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.081746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.081746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.081836 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.082064 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.082065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.082065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.082307 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:02.082461 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.082603 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.082751 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":79,"threads_started":1}
I20260619 16:54:02.083333 32483 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:02.090713 32483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0: Generated new TSK 0
I20260619 16:54:02.090870 32483 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:54:02.119899 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33707
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:02.226938 32498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:02.227131 32498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:02.227155 32498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:02.228312 32498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:02.228353 32498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:02.228374 32498 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:54:02.228391 32498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:02.229890 32498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.148.190:33707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:02.230130 32498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:02.230350 32498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:02.235049 32503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:02.235050 32506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:02.235095 32504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:02.235234 32498 server_base.cc:1034] running on GCE node
I20260619 16:54:02.235460 32498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:02.235721 32498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:02.236879 32498 hybrid_clock.cc:648] HybridClock initialized: now 1781888042236867 us; error 25 us; skew 500 ppm
Jun 19 16:54:02 dist-test-slave-xb9n krb5kdc[32381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888042, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:02.239337 32498 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:54:02.239750 32498 webserver.cc:460] Webserver started at http://127.30.148.129:35201/ using document root <none> and password file <none>
I20260619 16:54:02.239972 32498 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:02.240041 32498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:02.240191 32498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:02.241219 32498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5"
format_stamp: "Formatted at 2026-06-19 16:54:02 on dist-test-slave-xb9n"
server_key: "6e761ba3f337d289c3eaec83d8e6e46c"
server_key_iv: "c0353557db39347fd8b89d91b4b956ae"
server_key_version: "encryptionkey@0"
I20260619 16:54:02.241585 32498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5"
format_stamp: "Formatted at 2026-06-19 16:54:02 on dist-test-slave-xb9n"
server_key: "6e761ba3f337d289c3eaec83d8e6e46c"
server_key_iv: "c0353557db39347fd8b89d91b4b956ae"
server_key_version: "encryptionkey@0"
I20260619 16:54:02.242900 32498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:02.243676 32513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.244009 32498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:02.244107 32498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5"
format_stamp: "Formatted at 2026-06-19 16:54:02 on dist-test-slave-xb9n"
server_key: "6e761ba3f337d289c3eaec83d8e6e46c"
server_key_iv: "c0353557db39347fd8b89d91b4b956ae"
server_key_version: "encryptionkey@0"
I20260619 16:54:02.244195 32498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:02.254453 32498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:02.256230 32498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:02.256383 32498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:02.256840 32498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:02.256903 32498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.256944 32498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:02.256974 32498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.263794 32498 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:32981
I20260619 16:54:02.263837 32628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:32981 every 8 connection(s)
Jun 19 16:54:02 dist-test-slave-xb9n krb5kdc[32381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888042, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:02.264223 32498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:02.265091 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 32498
I20260619 16:54:02.265206 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:02.265442 31314 external_mini_cluster.cc:1428] Setting key 445c3189d91df8a3e9c0c6a9f2ccce46
I20260619 16:54:02.267927 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.257332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:44077 (local address 127.30.148.190:33707)
0619 16:54:02.257418 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.257421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.257657 (+ 236us) server_negotiation.cc:407] Connection header received
0619 16:54:02.258332 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.258333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.258343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.258387 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:02.259264 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.260065 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.261583 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.261797 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.261871 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.264733 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:02.264735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:02.264735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:02.264749 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:02.266516 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.266910 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.266911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.266912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.266975 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.267210 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.267211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.267212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.267357 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:02.267431 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.267799 (+ 368us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.267865 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 16:54:02.268196 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.257301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33707 (local address 127.30.148.129:44077)
0619 16:54:02.257505 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.257551 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.258214 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.258478 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.258483 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.258878 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:02.259163 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.259169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.260161 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.260162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.261492 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.261495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.261877 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.261879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.262011 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.262703 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:02.262714 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:02.264627 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:02.266613 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.266619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.266620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.266829 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.267049 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.267049 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.267049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.267144 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.267437 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:02.267444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:02.267640 (+ 196us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.267776 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.267948 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":104,"threads_started":1}
I20260619 16:54:02.268707 32629 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33707
I20260619 16:54:02.268832 32629 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:02.269023 32629 heartbeater.cc:511] Master 127.30.148.190:33707 requested a full tablet report, sending...
I20260619 16:54:02.269727 32425 ts_manager.cc:194] Registered new tserver with Master: 1bfce1c79e5a4056a5e1379cd6d1e6f5 (127.30.148.129:32981)
I20260619 16:54:02.270803 32425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:44077
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
I20260619 16:54:02.275739 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.269806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:54555 (local address 127.30.148.190:33707)
0619 16:54:02.269849 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.269851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.269955 (+ 104us) server_negotiation.cc:407] Connection header received
0619 16:54:02.270042 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.270043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.270055 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.270109 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:02.270442 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.270930 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.272016 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.272228 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.272301 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.273230 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:02.273231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:02.273231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:02.273238 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:02.274636 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.274985 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.274987 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.274987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.275038 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.275247 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.275248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.275248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.275357 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:02.275431 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.275579 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.275652 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:54:02.275766 32636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.269765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33707 (local address 127.30.148.129:54555)
0619 16:54:02.269903 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.269914 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.269969 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.270113 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.270114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.270310 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:02.270367 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.270369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.271231 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.271232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.271928 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.271930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.272309 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.272310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.272381 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.272834 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:02.272839 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:02.273118 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:02.274726 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.274727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.274727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.274898 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.275122 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.275122 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.275123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.275184 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.275423 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:02.275424 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:02.275466 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.275554 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.275652 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:54:02.289861 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33707
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:02.367550 32638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:02.367770 32638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:02.367811 32638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:02.368992 32638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:02.369060 32638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:02.369096 32638 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:54:02.369172 32638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:02.370791 32638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.148.190:33707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:02.371055 32638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:02.371279 32638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:54:02.376322 32638 server_base.cc:1034] running on GCE node
W20260619 16:54:02.376251 32644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:02.376250 32646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:02.376266 32643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:02.376739 32638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:02.376981 32638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:02.378137 32638 hybrid_clock.cc:648] HybridClock initialized: now 1781888042378131 us; error 40 us; skew 500 ppm
Jun 19 16:54:02 dist-test-slave-xb9n krb5kdc[32381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888042, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:02.380465 32638 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:54:02.380898 32638 webserver.cc:460] Webserver started at http://127.30.148.130:35169/ using document root <none> and password file <none>
I20260619 16:54:02.381090 32638 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:02.381129 32638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:02.381214 32638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:02.382225 32638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "f0729c76deb549ae997c759a0d50084b"
format_stamp: "Formatted at 2026-06-19 16:54:02 on dist-test-slave-xb9n"
server_key: "1a36d4ebb75776874bb8f5e1fa1c2ef0"
server_key_iv: "00265bf69a9d6ee6afa622161d5a281d"
server_key_version: "encryptionkey@0"
I20260619 16:54:02.382510 32638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "f0729c76deb549ae997c759a0d50084b"
format_stamp: "Formatted at 2026-06-19 16:54:02 on dist-test-slave-xb9n"
server_key: "1a36d4ebb75776874bb8f5e1fa1c2ef0"
server_key_iv: "00265bf69a9d6ee6afa622161d5a281d"
server_key_version: "encryptionkey@0"
I20260619 16:54:02.383692 32638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:02.384378 32653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.384528 32638 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:02.384620 32638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "f0729c76deb549ae997c759a0d50084b"
format_stamp: "Formatted at 2026-06-19 16:54:02 on dist-test-slave-xb9n"
server_key: "1a36d4ebb75776874bb8f5e1fa1c2ef0"
server_key_iv: "00265bf69a9d6ee6afa622161d5a281d"
server_key_version: "encryptionkey@0"
I20260619 16:54:02.384675 32638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:02.391526 32638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:02.393674 32638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:02.393788 32638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:02.394188 32638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:02.394227 32638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.394248 32638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:02.394315 32638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.400791 32638 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:41813
I20260619 16:54:02.400864 300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:41813 every 8 connection(s)
Jun 19 16:54:02 dist-test-slave-xb9n krb5kdc[32381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888042, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:02.401191 32638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:54:02.403870 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.394710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:38007 (local address 127.30.148.190:33707)
0619 16:54:02.394791 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.394794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.394929 (+ 135us) server_negotiation.cc:407] Connection header received
0619 16:54:02.395387 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.395389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.395401 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.395488 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:02.396299 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.397128 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.398703 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.399050 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.399136 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.401699 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:02.401700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:02.401700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:02.401706 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:02.402797 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.403146 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.403146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.403147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.403194 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.403403 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.403403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.403404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.403498 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:02.403563 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.403753 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.403809 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 16:54:02.404107 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.394642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33707 (local address 127.30.148.130:38007)
0619 16:54:02.394814 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.394850 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.395289 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.395538 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.395541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.395920 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:02.396191 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.396197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.397232 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.397233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.398598 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.398601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.399135 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.399136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.399230 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.399747 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:02.399756 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:02.401621 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:02.402863 (+ 1242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.402868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.402869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.403078 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.403257 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.403257 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.403258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.403350 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.403555 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:02.403559 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:02.403664 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.403743 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.403854 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260619 16:54:02.404585 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 32638
I20260619 16:54:02.404600 301 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33707
I20260619 16:54:02.404676 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:02.404678 301 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:02.404855 301 heartbeater.cc:511] Master 127.30.148.190:33707 requested a full tablet report, sending...
I20260619 16:54:02.404863 31314 external_mini_cluster.cc:1428] Setting key 301cfec19d7d5cad6192dfcbd03604da
I20260619 16:54:02.405236 32426 ts_manager.cc:194] Registered new tserver with Master: f0729c76deb549ae997c759a0d50084b (127.30.148.130:41813)
I20260619 16:54:02.405764 32426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:38007
I20260619 16:54:02.411857 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.405677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:44695 (local address 127.30.148.190:33707)
0619 16:54:02.405717 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.405719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.405999 (+ 280us) server_negotiation.cc:407] Connection header received
0619 16:54:02.406078 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.406079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.406091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.406141 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:02.406513 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.407120 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.408162 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.408366 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.408449 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.409386 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:02.409387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:02.409388 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:02.409394 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:02.410710 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.411061 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.411063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.411063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.411114 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.411317 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.411318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.411319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.411462 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:02.411543 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.411694 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.411767 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:02.411871 307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.405631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33707 (local address 127.30.148.130:44695)
0619 16:54:02.405939 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.405950 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.406010 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.406155 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.406155 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.406345 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:02.406420 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.406422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.407238 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.407239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.408094 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.408096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.408451 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.408451 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.408524 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.408967 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:02.408972 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:02.409253 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:02.410793 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.410794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.410795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.410968 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.411186 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.411187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.411187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.411244 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.411538 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:02.411539 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:02.411583 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.411670 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.411768 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":5,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:54:02.428249 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33707
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:02.507853 310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:02.508070 310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:02.508111 310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:02.509181 310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:02.509217 310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:02.509243 310 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:54:02.509260 310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:02.510601 310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.148.190:33707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:02.510784 310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:02.510944 310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:02.515930 315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:02.515923 316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:02.515923 318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:02.515950 310 server_base.cc:1034] running on GCE node
I20260619 16:54:02.516335 310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:02.516548 310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:02.517683 310 hybrid_clock.cc:648] HybridClock initialized: now 1781888042517666 us; error 32 us; skew 500 ppm
Jun 19 16:54:02 dist-test-slave-xb9n krb5kdc[32381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888042, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:02.520076 310 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:54:02.520475 310 webserver.cc:460] Webserver started at http://127.30.148.131:42411/ using document root <none> and password file <none>
I20260619 16:54:02.520685 310 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:02.520728 310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:02.520849 310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:02.521875 310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "cdd2a647857143ab8f752d2a17b697c4"
format_stamp: "Formatted at 2026-06-19 16:54:02 on dist-test-slave-xb9n"
server_key: "9445e81a1b65d1998856d1af2221feb7"
server_key_iv: "8fdde68023ab49cf19a47b1b229d73f5"
server_key_version: "encryptionkey@0"
I20260619 16:54:02.522245 310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "cdd2a647857143ab8f752d2a17b697c4"
format_stamp: "Formatted at 2026-06-19 16:54:02 on dist-test-slave-xb9n"
server_key: "9445e81a1b65d1998856d1af2221feb7"
server_key_iv: "8fdde68023ab49cf19a47b1b229d73f5"
server_key_version: "encryptionkey@0"
I20260619 16:54:02.523490 310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:02.524190 325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.524405 310 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:02.524497 310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "cdd2a647857143ab8f752d2a17b697c4"
format_stamp: "Formatted at 2026-06-19 16:54:02 on dist-test-slave-xb9n"
server_key: "9445e81a1b65d1998856d1af2221feb7"
server_key_iv: "8fdde68023ab49cf19a47b1b229d73f5"
server_key_version: "encryptionkey@0"
I20260619 16:54:02.524574 310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:02.545179 310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:02.547703 310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:02.547868 310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:02.548383 310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:02.548442 310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.548476 310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:02.548503 310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.554876 310 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:46191
I20260619 16:54:02.554947 440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:46191 every 8 connection(s)
I20260619 16:54:02.555212 310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:54:02 dist-test-slave-xb9n krb5kdc[32381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888042, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:02.558449 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.548824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:53865 (local address 127.30.148.190:33707)
0619 16:54:02.548864 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.548866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.549086 (+ 220us) server_negotiation.cc:407] Connection header received
0619 16:54:02.549670 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.549672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.549685 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.549762 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:02.550652 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.551488 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.553062 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.553418 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.553501 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.555986 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:02.555987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:02.555988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:02.555994 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:02.557220 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.557631 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.557632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.557632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.557685 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.557956 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.557956 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.557956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.558047 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:02.558092 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.558330 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.558387 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:54:02.558624 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.548815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33707 (local address 127.30.148.131:53865)
0619 16:54:02.548966 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.548999 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.549564 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.549855 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.549859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.550228 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:02.550516 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.550522 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.551596 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.551597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.552962 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.552964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.553508 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.553509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.553647 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.554157 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:02.554170 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:02.555902 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:02.557293 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.557299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.557301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.557563 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.557746 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.557747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.557747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.557897 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.558099 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:02.558103 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:02.558209 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.558314 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.558426 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260619 16:54:02.559098 441 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33707
I20260619 16:54:02.559180 441 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:02.559356 441 heartbeater.cc:511] Master 127.30.148.190:33707 requested a full tablet report, sending...
I20260619 16:54:02.559787 32425 ts_manager.cc:194] Registered new tserver with Master: cdd2a647857143ab8f752d2a17b697c4 (127.30.148.131:46191)
I20260619 16:54:02.560211 32425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:53865
I20260619 16:54:02.562839 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 310
I20260619 16:54:02.562913 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:02.563102 31314 external_mini_cluster.cc:1428] Setting key be6fc230314ffbb3a27cfb85080bd49d
I20260619 16:54:02.564225 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:54:02.566011 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.560160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33707 (local address 127.30.148.131:48649)
0619 16:54:02.560317 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.560327 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.560410 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.560539 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.560540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.560713 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:02.560762 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.560764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.561425 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.561426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.561937 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.561938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.562318 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.562319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.562394 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.562840 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:02.562844 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:02.563171 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:02.565062 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.565063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.565063 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.565222 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.565439 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.565440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.565440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.565497 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.565783 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:02.565784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:02.565824 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.565874 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.565946 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":63,"threads_started":1}
I20260619 16:54:02.566100 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.560214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:48649 (local address 127.30.148.190:33707)
0619 16:54:02.560260 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.560263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.560401 (+ 138us) server_negotiation.cc:407] Connection header received
0619 16:54:02.560469 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.560470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.560480 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.560535 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:02.560837 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.561304 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.562036 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.562235 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.562316 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.563278 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:02.563279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:02.563279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:02.563286 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:02.564980 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.565303 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.565304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.565305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.565385 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.565558 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.565559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.565559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.565722 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:02.565794 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.565897 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.565976 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Jun 19 16:54:02 dist-test-slave-xb9n krb5kdc[32381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 16:54:02 dist-test-slave-xb9n krb5kdc[32381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:02.587126 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.580203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33910 (local address 127.30.148.190:33707)
0619 16:54:02.580268 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.580270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.580281 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:54:02.580326 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.580327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.580336 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.580399 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:02.580945 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.581546 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.582284 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.582492 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.582576 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.584776 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:02.584778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:02.584778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:02.584784 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:02.586241 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.586537 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.586538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.586538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.586585 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.586744 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.586744 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.586744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.586840 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:02.586918 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.587005 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.587062 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 16:54:02.588850 32425 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:02.590459 32425 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:02.596668 458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.592836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:46191 (local address 127.0.0.1:53358)
0619 16:54:02.592945 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.592977 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.593062 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.593324 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.593327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.593329 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:02.593423 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.593425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.594476 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.594478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.595451 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.595453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.596364 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.596365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.596453 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.596467 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.596551 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.596598 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":72,"threads_started":1}
I20260619 16:54:02.596710 459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.592930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53358 (local address 127.30.148.131:46191)
0619 16:54:02.593102 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.593108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.593120 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:54:02.593157 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.593159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.593174 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.593246 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:02.593493 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.594391 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.595547 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.596277 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.596360 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.596417 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.596579 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.596618 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":78,"threads_started":1}
I20260619 16:54:02.597332 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.593195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:41813 (local address 127.0.0.1:59224)
0619 16:54:02.593297 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.593306 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.593345 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.593843 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.593844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.593846 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:02.593888 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.593890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.595186 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.595187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.596078 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.596080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.597070 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.597071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.597132 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.597135 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.597223 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.597261 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":8,"thread_start_us":75,"threads_started":1}
I20260619 16:54:02.597393 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.593290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.30.148.130:41813)
0619 16:54:02.593634 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.593638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.593648 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:02.593685 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.593687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.593702 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.593781 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:02.593973 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.595097 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.596166 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.596906 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.596974 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.597036 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.597240 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.597274 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":116,"threads_started":1}
I20260619 16:54:02.597641 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.593055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:32981 (local address 127.0.0.1:51450)
0619 16:54:02.593142 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.593152 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.593200 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.593757 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.593758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.593760 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:02.593810 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.593812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.595177 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.595178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.596292 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.596294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.597419 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.597421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.597481 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.597484 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.597513 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.597552 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":27,"thread_start_us":51,"threads_started":1}
I20260619 16:54:02.597596 375 tablet_service.cc:1467] Processing CreateTablet for tablet 939ad9abbc9c4aaea983ac27e0bb3095 (DEFAULT_TABLE table=test-table [id=7b4c6349d47442f1afe22ebfe60bb408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:02.597999 375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 939ad9abbc9c4aaea983ac27e0bb3095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:02.598030 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.593433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51450 (local address 127.30.148.129:32981)
0619 16:54:02.593587 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.593592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.593601 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:54:02.593640 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.593642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.593657 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.593885 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:02.593972 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.595080 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.596381 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.597295 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.597809 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.597866 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.597890 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.597933 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260619 16:54:02.598220 32703 tablet_service.cc:1467] Processing CreateTablet for tablet 939ad9abbc9c4aaea983ac27e0bb3095 (DEFAULT_TABLE table=test-table [id=7b4c6349d47442f1afe22ebfe60bb408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:02.598565 32703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 939ad9abbc9c4aaea983ac27e0bb3095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:02.598551 32563 tablet_service.cc:1467] Processing CreateTablet for tablet 939ad9abbc9c4aaea983ac27e0bb3095 (DEFAULT_TABLE table=test-table [id=7b4c6349d47442f1afe22ebfe60bb408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:02.598881 32563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 939ad9abbc9c4aaea983ac27e0bb3095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:02.601279 464 tablet_bootstrap.cc:492] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4: Bootstrap starting.
I20260619 16:54:02.601698 465 tablet_bootstrap.cc:492] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b: Bootstrap starting.
I20260619 16:54:02.602279 466 tablet_bootstrap.cc:492] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Bootstrap starting.
I20260619 16:54:02.602326 465 tablet_bootstrap.cc:654] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:02.602326 464 tablet_bootstrap.cc:654] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:02.602672 464 log.cc:824] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:02.602671 465 log.cc:824] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:02.602926 466 tablet_bootstrap.cc:654] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:02.603273 466 log.cc:824] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:02.603430 464 tablet_bootstrap.cc:492] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4: No bootstrap required, opened a new log
I20260619 16:54:02.603503 464 ts_tablet_manager.cc:1397] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:54:02.603654 465 tablet_bootstrap.cc:492] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b: No bootstrap required, opened a new log
I20260619 16:54:02.603713 465 ts_tablet_manager.cc:1397] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:02.604002 466 tablet_bootstrap.cc:492] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: No bootstrap required, opened a new log
I20260619 16:54:02.604044 466 ts_tablet_manager.cc:1397] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:02.605559 464 raft_consensus.cc:348] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } }
I20260619 16:54:02.605719 464 raft_consensus.cc:374] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:02.605752 464 raft_consensus.cc:729] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd2a647857143ab8f752d2a17b697c4, State: Initialized, Role: FOLLOWER
I20260619 16:54:02.605667 465 raft_consensus.cc:348] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } }
I20260619 16:54:02.605787 465 raft_consensus.cc:374] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:02.605815 465 raft_consensus.cc:729] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0729c76deb549ae997c759a0d50084b, State: Initialized, Role: FOLLOWER
I20260619 16:54:02.605796 466 raft_consensus.cc:348] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } }
I20260619 16:54:02.605911 465 consensus_queue.cc:260] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } }
I20260619 16:54:02.605872 464 consensus_queue.cc:260] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } }
I20260619 16:54:02.605955 466 raft_consensus.cc:374] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:02.605980 466 raft_consensus.cc:729] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bfce1c79e5a4056a5e1379cd6d1e6f5, State: Initialized, Role: FOLLOWER
I20260619 16:54:02.606081 466 consensus_queue.cc:260] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } }
I20260619 16:54:02.606151 464 ts_tablet_manager.cc:1428] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:54:02.606150 465 ts_tablet_manager.cc:1428] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 16:54:02.606259 441 heartbeater.cc:503] Master 127.30.148.190:33707 was elected leader, sending a full tablet report...
I20260619 16:54:02.606282 301 heartbeater.cc:503] Master 127.30.148.190:33707 was elected leader, sending a full tablet report...
I20260619 16:54:02.606319 466 ts_tablet_manager.cc:1428] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:54:02.606451 32629 heartbeater.cc:503] Master 127.30.148.190:33707 was elected leader, sending a full tablet report...
W20260619 16:54:02.651779 302 tablet.cc:2369] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:54:02.684612 472 raft_consensus.cc:482] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:02.684724 472 raft_consensus.cc:504] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } }
I20260619 16:54:02.685034 472 leader_election.cc:288] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdd2a647857143ab8f752d2a17b697c4 (127.30.148.131:46191), f0729c76deb549ae997c759a0d50084b (127.30.148.130:41813)
I20260619 16:54:02.688115 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.685590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:38027 (local address 127.30.148.130:41813)
0619 16:54:02.685660 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.685662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.685670 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:02.685696 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.685697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.685705 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.685771 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:02.685897 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.686364 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.687407 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.687761 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.687849 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.687858 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.688011 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.688046 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 16:54:02.688131 473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.685339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:41813 (local address 127.30.148.129:38027)
0619 16:54:02.685548 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.685558 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.685606 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.685770 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.685770 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.685772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:02.685833 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.685835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.686452 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.686453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.687318 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.687320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.687858 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.687859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.687932 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.687940 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.688006 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.688046 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":129,"threads_started":1}
I20260619 16:54:02.688190 459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.685295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:50111 (local address 127.30.148.131:46191)
0619 16:54:02.685343 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.685345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.685396 (+ 51us) server_negotiation.cc:407] Connection header received
0619 16:54:02.685424 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.685425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.685434 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.685508 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:02.685712 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.686278 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.687307 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.687843 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.687910 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.687918 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.688070 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.688121 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 16:54:02.688560 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.685209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:46191 (local address 127.30.148.129:50111)
0619 16:54:02.685264 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.685276 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.685318 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.685529 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.685530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.685533 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:02.685626 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.685628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.686354 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.686355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.687221 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.687223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.687908 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.687909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.687995 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.687998 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.688065 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.688101 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":29}
I20260619 16:54:02.688575 32723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "939ad9abbc9c4aaea983ac27e0bb3095" candidate_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0729c76deb549ae997c759a0d50084b" is_pre_election: true
I20260619 16:54:02.688733 32723 raft_consensus.cc:2454] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bfce1c79e5a4056a5e1379cd6d1e6f5 in term 0.
I20260619 16:54:02.688890 379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "939ad9abbc9c4aaea983ac27e0bb3095" candidate_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd2a647857143ab8f752d2a17b697c4" is_pre_election: true
I20260619 16:54:02.688944 32517 leader_election.cc:302] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bfce1c79e5a4056a5e1379cd6d1e6f5, f0729c76deb549ae997c759a0d50084b; no voters:
I20260619 16:54:02.689021 379 raft_consensus.cc:2454] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bfce1c79e5a4056a5e1379cd6d1e6f5 in term 0.
I20260619 16:54:02.689070 472 raft_consensus.cc:2781] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:02.689121 472 raft_consensus.cc:482] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:02.689149 472 raft_consensus.cc:3037] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:02.689931 472 raft_consensus.cc:504] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } }
I20260619 16:54:02.690023 472 leader_election.cc:288] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [CANDIDATE]: Term 1 election: Requested vote from peers cdd2a647857143ab8f752d2a17b697c4 (127.30.148.131:46191), f0729c76deb549ae997c759a0d50084b (127.30.148.130:41813)
I20260619 16:54:02.690212 379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "939ad9abbc9c4aaea983ac27e0bb3095" candidate_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd2a647857143ab8f752d2a17b697c4"
I20260619 16:54:02.690204 32723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "939ad9abbc9c4aaea983ac27e0bb3095" candidate_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0729c76deb549ae997c759a0d50084b"
I20260619 16:54:02.690291 379 raft_consensus.cc:3037] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:02.690287 32723 raft_consensus.cc:3037] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:02.690972 32723 raft_consensus.cc:2454] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bfce1c79e5a4056a5e1379cd6d1e6f5 in term 1.
I20260619 16:54:02.690984 379 raft_consensus.cc:2454] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bfce1c79e5a4056a5e1379cd6d1e6f5 in term 1.
I20260619 16:54:02.691130 32517 leader_election.cc:302] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bfce1c79e5a4056a5e1379cd6d1e6f5, cdd2a647857143ab8f752d2a17b697c4; no voters:
I20260619 16:54:02.691210 472 raft_consensus.cc:2781] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:02.691359 472 raft_consensus.cc:686] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 1 LEADER]: Becoming Leader. State: Replica: 1bfce1c79e5a4056a5e1379cd6d1e6f5, State: Running, Role: LEADER
I20260619 16:54:02.691457 472 consensus_queue.cc:237] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } }
I20260619 16:54:02.692237 32424 catalog_manager.cc:5495] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bfce1c79e5a4056a5e1379cd6d1e6f5 (127.30.148.129). New cstate: current_term: 1 leader_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } health_report { overall_health: UNKNOWN } } }
I20260619 16:54:02.712298 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.709954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.30.148.129:32981)
0619 16:54:02.710019 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.710023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.710035 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:54:02.710110 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.710112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.710125 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.710180 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:02.710310 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.710729 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.711426 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.711592 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.711639 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.711804 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:02.711887 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:02.711942 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.711989 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.712005 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 16:54:02.713979 32723 raft_consensus.cc:1261] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b [term 1 FOLLOWER]: Refusing update from remote peer 1bfce1c79e5a4056a5e1379cd6d1e6f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:02.714025 379 raft_consensus.cc:1261] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4 [term 1 FOLLOWER]: Refusing update from remote peer 1bfce1c79e5a4056a5e1379cd6d1e6f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:02.714306 472 consensus_queue.cc:1035] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:02.714452 474 consensus_queue.cc:1035] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:02.717585 479 mvcc.cc:205] Tried to move back new op lower bound from 7298613422953656320 to 7298613422864826368. Current Snapshot: MvccSnapshot[applied={T|T < 7298613422953656320}]
I20260619 16:54:02.720168 32426 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:02.720263 32426 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:02.720997 32426 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0: Sending DeleteTablet for 3 replicas of tablet 939ad9abbc9c4aaea983ac27e0bb3095
I20260619 16:54:02.721310 375 tablet_service.cc:1514] Processing DeleteTablet for tablet 939ad9abbc9c4aaea983ac27e0bb3095 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:02 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:53358
I20260619 16:54:02.721343 32703 tablet_service.cc:1514] Processing DeleteTablet for tablet 939ad9abbc9c4aaea983ac27e0bb3095 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:02 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:59224
I20260619 16:54:02.721338 32563 tablet_service.cc:1514] Processing DeleteTablet for tablet 939ad9abbc9c4aaea983ac27e0bb3095 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:02 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:51450
I20260619 16:54:02.721573 488 tablet_replica.cc:330] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: stopping tablet replica
I20260619 16:54:02.721670 488 raft_consensus.cc:2229] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:02.721786 486 tablet_replica.cc:330] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4: stopping tablet replica
I20260619 16:54:02.721796 488 raft_consensus.cc:2258] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:02.722038 32426 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:02.722196 32426 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:02.722203 488 ts_tablet_manager.cc:1905] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:02.722426 486 raft_consensus.cc:2229] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:02.722568 486 raft_consensus.cc:2258] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:02.723520 32563 tablet_service.cc:1467] Processing CreateTablet for tablet 03b6f0f0d64c4eb985945dae5165ea8b (DEFAULT_TABLE table=test-table [id=c605b9fa21f241f5bf9aad941c331194]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:02.723623 32563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03b6f0f0d64c4eb985945dae5165ea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:02.723598 32703 tablet_service.cc:1467] Processing CreateTablet for tablet 03b6f0f0d64c4eb985945dae5165ea8b (DEFAULT_TABLE table=test-table [id=c605b9fa21f241f5bf9aad941c331194]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:02.723718 32703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03b6f0f0d64c4eb985945dae5165ea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:02.723935 375 tablet_service.cc:1467] Processing CreateTablet for tablet 03b6f0f0d64c4eb985945dae5165ea8b (DEFAULT_TABLE table=test-table [id=c605b9fa21f241f5bf9aad941c331194]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:02.724031 375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03b6f0f0d64c4eb985945dae5165ea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:02.725073 465 tablet_bootstrap.cc:492] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b: Bootstrap starting.
I20260619 16:54:02.725101 466 tablet_bootstrap.cc:492] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Bootstrap starting.
I20260619 16:54:02.725301 464 tablet_bootstrap.cc:492] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4: Bootstrap starting.
I20260619 16:54:02.725412 488 ts_tablet_manager.cc:1918] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:02.725476 488 log.cc:1196] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/wal/wals/939ad9abbc9c4aaea983ac27e0bb3095
I20260619 16:54:02.725566 466 tablet_bootstrap.cc:654] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:02.725687 464 tablet_bootstrap.cc:654] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:02.725721 488 ts_tablet_manager.cc:1939] T 939ad9abbc9c4aaea983ac27e0bb3095 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Deleting consensus metadata
I20260619 16:54:02.726090 32413 catalog_manager.cc:4841] TS 1bfce1c79e5a4056a5e1379cd6d1e6f5 (127.30.148.129:32981): tablet 939ad9abbc9c4aaea983ac27e0bb3095 (table test-table [id=7b4c6349d47442f1afe22ebfe60bb408]) successfully deleted
I20260619 16:54:02.726277 486 ts_tablet_manager.cc:1905] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:02.727376 486 ts_tablet_manager.cc:1918] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:02.727463 486 log.cc:1196] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/939ad9abbc9c4aaea983ac27e0bb3095
I20260619 16:54:02.727718 486 ts_tablet_manager.cc:1939] T 939ad9abbc9c4aaea983ac27e0bb3095 P cdd2a647857143ab8f752d2a17b697c4: Deleting consensus metadata
I20260619 16:54:02.728062 487 tablet_replica.cc:330] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b: stopping tablet replica
I20260619 16:54:02.728113 32413 catalog_manager.cc:4841] TS cdd2a647857143ab8f752d2a17b697c4 (127.30.148.131:46191): tablet 939ad9abbc9c4aaea983ac27e0bb3095 (table test-table [id=7b4c6349d47442f1afe22ebfe60bb408]) successfully deleted
I20260619 16:54:02.728173 487 raft_consensus.cc:2229] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:02.728282 487 raft_consensus.cc:2258] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:02.728575 465 tablet_bootstrap.cc:654] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:02.729137 465 tablet_bootstrap.cc:492] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b: No bootstrap required, opened a new log
I20260619 16:54:02.729174 487 ts_tablet_manager.cc:1905] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:02.729187 465 ts_tablet_manager.cc:1397] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 16:54:02.729383 464 tablet_bootstrap.cc:492] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4: No bootstrap required, opened a new log
I20260619 16:54:02.729435 464 ts_tablet_manager.cc:1397] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 16:54:02.729393 465 raft_consensus.cc:348] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.729470 465 raft_consensus.cc:374] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:02.729499 465 raft_consensus.cc:729] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0729c76deb549ae997c759a0d50084b, State: Initialized, Role: FOLLOWER
I20260619 16:54:02.729540 465 consensus_queue.cc:260] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.729583 464 raft_consensus.cc:348] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.729637 465 ts_tablet_manager.cc:1428] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.729650 464 raft_consensus.cc:374] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:02.729671 464 raft_consensus.cc:729] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd2a647857143ab8f752d2a17b697c4, State: Initialized, Role: FOLLOWER
I20260619 16:54:02.729710 464 consensus_queue.cc:260] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.729796 464 ts_tablet_manager.cc:1428] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.730063 466 tablet_bootstrap.cc:492] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5: No bootstrap required, opened a new log
I20260619 16:54:02.730113 466 ts_tablet_manager.cc:1397] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260619 16:54:02.730244 466 raft_consensus.cc:348] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.730314 466 raft_consensus.cc:374] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:02.730338 466 raft_consensus.cc:729] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bfce1c79e5a4056a5e1379cd6d1e6f5, State: Initialized, Role: FOLLOWER
I20260619 16:54:02.730396 466 consensus_queue.cc:260] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.730513 466 ts_tablet_manager.cc:1428] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.730681 487 ts_tablet_manager.cc:1918] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:02.730760 487 log.cc:1196] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/939ad9abbc9c4aaea983ac27e0bb3095
I20260619 16:54:02.731012 487 ts_tablet_manager.cc:1939] T 939ad9abbc9c4aaea983ac27e0bb3095 P f0729c76deb549ae997c759a0d50084b: Deleting consensus metadata
I20260619 16:54:02.731369 32413 catalog_manager.cc:4841] TS f0729c76deb549ae997c759a0d50084b (127.30.148.130:41813): tablet 939ad9abbc9c4aaea983ac27e0bb3095 (table test-table [id=7b4c6349d47442f1afe22ebfe60bb408]) successfully deleted
W20260619 16:54:02.764856 32630 tablet.cc:2369] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:54:02.805948 442 tablet.cc:2369] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:54:02.846232 470 raft_consensus.cc:482] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:02.846334 470 raft_consensus.cc:504] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.846683 470 leader_election.cc:288] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bfce1c79e5a4056a5e1379cd6d1e6f5 (127.30.148.129:32981), cdd2a647857143ab8f752d2a17b697c4 (127.30.148.131:46191)
I20260619 16:54:02.849792 459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.847178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:40801 (local address 127.30.148.131:46191)
0619 16:54:02.847232 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.847234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.847263 (+ 29us) server_negotiation.cc:407] Connection header received
0619 16:54:02.847327 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.847328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.847339 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.847389 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:02.847577 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.848079 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.849031 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.849412 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.849496 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.849506 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.849671 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.849706 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 16:54:02.849807 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.846973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:46805 (local address 127.30.148.129:32981)
0619 16:54:02.847154 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.847156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.847167 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:54:02.847196 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.847197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.847208 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.847266 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:02.847441 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.848054 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.849020 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.849440 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.849537 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.849546 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.849704 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.849737 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260619 16:54:02.849825 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.847034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:46191 (local address 127.30.148.130:40801)
0619 16:54:02.847189 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.847198 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.847276 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.847391 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.847391 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.847392 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:02.847480 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.847482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.848166 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.848167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.848930 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.848932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.849505 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.849506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.849585 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.849589 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.849650 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.849692 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260619 16:54:02.849838 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.846901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:32981 (local address 127.30.148.130:46805)
0619 16:54:02.846966 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.846979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.847040 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.847285 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.847286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.847288 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:02.847365 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.847367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.848139 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.848140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.848914 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.848916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.849537 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.849538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.849627 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.849630 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.849699 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.849736 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":19}
I20260619 16:54:02.850106 379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03b6f0f0d64c4eb985945dae5165ea8b" candidate_uuid: "f0729c76deb549ae997c759a0d50084b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd2a647857143ab8f752d2a17b697c4" is_pre_election: true
I20260619 16:54:02.850219 379 raft_consensus.cc:2454] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0729c76deb549ae997c759a0d50084b in term 0.
I20260619 16:54:02.850188 32583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03b6f0f0d64c4eb985945dae5165ea8b" candidate_uuid: "f0729c76deb549ae997c759a0d50084b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" is_pre_election: true
I20260619 16:54:02.850307 32583 raft_consensus.cc:2454] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0729c76deb549ae997c759a0d50084b in term 0.
I20260619 16:54:02.850392 32657 leader_election.cc:302] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cdd2a647857143ab8f752d2a17b697c4, f0729c76deb549ae997c759a0d50084b; no voters:
I20260619 16:54:02.850538 470 raft_consensus.cc:2781] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:02.850605 470 raft_consensus.cc:482] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:02.850634 470 raft_consensus.cc:3037] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:02.851305 470 raft_consensus.cc:504] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.851441 470 leader_election.cc:288] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [CANDIDATE]: Term 1 election: Requested vote from peers 1bfce1c79e5a4056a5e1379cd6d1e6f5 (127.30.148.129:32981), cdd2a647857143ab8f752d2a17b697c4 (127.30.148.131:46191)
I20260619 16:54:02.851586 32583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03b6f0f0d64c4eb985945dae5165ea8b" candidate_uuid: "f0729c76deb549ae997c759a0d50084b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5"
I20260619 16:54:02.851624 379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03b6f0f0d64c4eb985945dae5165ea8b" candidate_uuid: "f0729c76deb549ae997c759a0d50084b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd2a647857143ab8f752d2a17b697c4"
I20260619 16:54:02.851653 32583 raft_consensus.cc:3037] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:02.851711 379 raft_consensus.cc:3037] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:02.852301 379 raft_consensus.cc:2454] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0729c76deb549ae997c759a0d50084b in term 1.
I20260619 16:54:02.852355 32583 raft_consensus.cc:2454] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0729c76deb549ae997c759a0d50084b in term 1.
I20260619 16:54:02.852517 32657 leader_election.cc:302] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bfce1c79e5a4056a5e1379cd6d1e6f5, f0729c76deb549ae997c759a0d50084b; no voters:
I20260619 16:54:02.852607 470 raft_consensus.cc:2781] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:02.852732 470 raft_consensus.cc:686] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 1 LEADER]: Becoming Leader. State: Replica: f0729c76deb549ae997c759a0d50084b, State: Running, Role: LEADER
I20260619 16:54:02.852798 470 consensus_queue.cc:237] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.853521 32426 catalog_manager.cc:5495] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b reported cstate change: term changed from 0 to 1, leader changed from <none> to f0729c76deb549ae997c759a0d50084b (127.30.148.130). New cstate: current_term: 1 leader_uuid: "f0729c76deb549ae997c759a0d50084b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } health_report { overall_health: UNKNOWN } } }
Jun 19 16:54:02 dist-test-slave-xb9n krb5kdc[32381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888042, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:02.896687 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.890070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.30.148.190:33707)
0619 16:54:02.890138 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.890139 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.890193 (+ 54us) server_negotiation.cc:407] Connection header received
0619 16:54:02.890250 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.890251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.890259 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.890327 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:02.890765 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.891232 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.891929 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.892099 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.892145 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.894106 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:02.894107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:02.894107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:02.894118 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:02.895483 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.895887 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.895888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.895888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.895945 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.896137 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.896138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.896139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.896248 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:02.896324 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.896473 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.896576 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":45}
I20260619 16:54:02.896761 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.889950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33707 (local address 127.0.0.1:33926)
0619 16:54:02.890125 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.890137 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.890194 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.890356 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.890357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.890654 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:02.890700 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.890702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.891341 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.891342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.891857 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.891859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.892160 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.892161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.892199 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.892666 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:02.892670 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:02.894006 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:02.895605 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.895611 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.895613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.895808 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.896011 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:02.896012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:02.896012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:02.896070 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:02.896315 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:02.896321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:02.896365 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.896451 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.896632 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":80,"thread_start_us":57,"threads_started":1}
I20260619 16:54:02.898531 32426 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:33926:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 16:54:02.898762 32426 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:02.900122 32563 tablet_service.cc:1467] Processing CreateTablet for tablet dbc089f7bc3b45e2a51ab07f797ac213 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=36237787564b4d20816b664c20196840]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:54:02.900180 32703 tablet_service.cc:1467] Processing CreateTablet for tablet dbc089f7bc3b45e2a51ab07f797ac213 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=36237787564b4d20816b664c20196840]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:54:02.900192 375 tablet_service.cc:1467] Processing CreateTablet for tablet dbc089f7bc3b45e2a51ab07f797ac213 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=36237787564b4d20816b664c20196840]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:54:02.900283 32563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbc089f7bc3b45e2a51ab07f797ac213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:02.900296 32703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbc089f7bc3b45e2a51ab07f797ac213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:02.900365 375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbc089f7bc3b45e2a51ab07f797ac213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:02.901496 464 tablet_bootstrap.cc:492] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: Bootstrap starting.
I20260619 16:54:02.901755 465 tablet_bootstrap.cc:492] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b: Bootstrap starting.
I20260619 16:54:02.901959 464 tablet_bootstrap.cc:654] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:02.902150 465 tablet_bootstrap.cc:654] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:02.902455 466 tablet_bootstrap.cc:492] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Bootstrap starting.
I20260619 16:54:02.902634 464 tablet_bootstrap.cc:492] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: No bootstrap required, opened a new log
I20260619 16:54:02.902688 464 ts_tablet_manager.cc:1397] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:02.902756 466 tablet_bootstrap.cc:654] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:02.902838 465 tablet_bootstrap.cc:492] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b: No bootstrap required, opened a new log
I20260619 16:54:02.902879 465 ts_tablet_manager.cc:1397] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:02.902848 464 raft_consensus.cc:348] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.902916 464 raft_consensus.cc:374] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:02.902941 464 raft_consensus.cc:729] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd2a647857143ab8f752d2a17b697c4, State: Initialized, Role: FOLLOWER
I20260619 16:54:02.902983 464 consensus_queue.cc:260] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.903017 465 raft_consensus.cc:348] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.903074 465 raft_consensus.cc:374] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:02.903093 465 raft_consensus.cc:729] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0729c76deb549ae997c759a0d50084b, State: Initialized, Role: FOLLOWER
I20260619 16:54:02.903092 464 ts_tablet_manager.cc:1428] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:02.903139 465 consensus_queue.cc:260] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.903224 465 ts_tablet_manager.cc:1428] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.903270 466 tablet_bootstrap.cc:492] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: No bootstrap required, opened a new log
I20260619 16:54:02.903247 471 tablet_replica.cc:439] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } }
I20260619 16:54:02.903314 466 ts_tablet_manager.cc:1397] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:02.903322 471 tablet_replica.cc:442] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:02.903416 466 raft_consensus.cc:348] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.903496 466 raft_consensus.cc:374] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:02.903517 466 raft_consensus.cc:729] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bfce1c79e5a4056a5e1379cd6d1e6f5, State: Initialized, Role: FOLLOWER
I20260619 16:54:02.903556 466 consensus_queue.cc:260] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.903661 466 ts_tablet_manager.cc:1428] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:02.903689 472 tablet_replica.cc:439] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } }
I20260619 16:54:02.903760 472 tablet_replica.cc:442] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:02.904347 470 tablet_replica.cc:439] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } }
I20260619 16:54:02.904450 470 tablet_replica.cc:442] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:02.915508 471 raft_consensus.cc:482] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:02.915561 471 raft_consensus.cc:504] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.915783 471 leader_election.cc:288] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bfce1c79e5a4056a5e1379cd6d1e6f5 (127.30.148.129:32981), f0729c76deb549ae997c759a0d50084b (127.30.148.130:41813)
I20260619 16:54:02.918684 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.915960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:32981 (local address 127.30.148.131:35999)
0619 16:54:02.916019 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.916031 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.916074 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.916233 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.916243 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.916245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:02.916306 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.916307 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.917124 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.917125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.917907 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.917909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.918416 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.918417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.918495 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.918498 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.918584 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.918620 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 16:54:02.918684 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.916066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:35999 (local address 127.30.148.129:32981)
0619 16:54:02.916125 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.916127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.916134 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:54:02.916156 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.916157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.916166 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.916217 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:02.916388 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.917039 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.918001 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.918329 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.918416 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.918424 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.918592 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.918627 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 16:54:02.919036 32583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbc089f7bc3b45e2a51ab07f797ac213" candidate_uuid: "cdd2a647857143ab8f752d2a17b697c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" is_pre_election: true
I20260619 16:54:02.919139 32583 raft_consensus.cc:2454] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdd2a647857143ab8f752d2a17b697c4 in term 0.
I20260619 16:54:02.919202 504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.916049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:41813 (local address 127.30.148.131:58231)
0619 16:54:02.916198 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.916209 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.916269 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.916398 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.916398 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.916400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:02.916474 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.916476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.917229 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.917230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.918295 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.918297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.918954 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.918955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.919032 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.919036 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.919083 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.919122 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260619 16:54:02.919237 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.916177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:58231 (local address 127.30.148.130:41813)
0619 16:54:02.916221 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.916224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.916253 (+ 29us) server_negotiation.cc:407] Connection header received
0619 16:54:02.916333 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.916333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.916341 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.916395 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:02.916540 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.917151 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.918387 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.918769 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.918841 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.918850 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.919103 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.919165 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:54:02.919319 329 leader_election.cc:302] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bfce1c79e5a4056a5e1379cd6d1e6f5, cdd2a647857143ab8f752d2a17b697c4; no voters:
I20260619 16:54:02.919450 471 raft_consensus.cc:2781] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:02.919498 471 raft_consensus.cc:482] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:02.919520 471 raft_consensus.cc:3037] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:02.919589 32723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbc089f7bc3b45e2a51ab07f797ac213" candidate_uuid: "cdd2a647857143ab8f752d2a17b697c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0729c76deb549ae997c759a0d50084b" is_pre_election: true
I20260619 16:54:02.919663 32723 raft_consensus.cc:2454] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdd2a647857143ab8f752d2a17b697c4 in term 0.
I20260619 16:54:02.920176 471 raft_consensus.cc:504] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.920332 471 leader_election.cc:288] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [CANDIDATE]: Term 1 election: Requested vote from peers 1bfce1c79e5a4056a5e1379cd6d1e6f5 (127.30.148.129:32981), f0729c76deb549ae997c759a0d50084b (127.30.148.130:41813)
I20260619 16:54:02.920481 32583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbc089f7bc3b45e2a51ab07f797ac213" candidate_uuid: "cdd2a647857143ab8f752d2a17b697c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5"
I20260619 16:54:02.920506 32723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbc089f7bc3b45e2a51ab07f797ac213" candidate_uuid: "cdd2a647857143ab8f752d2a17b697c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0729c76deb549ae997c759a0d50084b"
I20260619 16:54:02.920545 32583 raft_consensus.cc:3037] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:02.920565 32723 raft_consensus.cc:3037] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:02.921051 32583 raft_consensus.cc:2454] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdd2a647857143ab8f752d2a17b697c4 in term 1.
I20260619 16:54:02.921242 32723 raft_consensus.cc:2454] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdd2a647857143ab8f752d2a17b697c4 in term 1.
I20260619 16:54:02.921248 329 leader_election.cc:302] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bfce1c79e5a4056a5e1379cd6d1e6f5, cdd2a647857143ab8f752d2a17b697c4; no voters:
I20260619 16:54:02.921329 471 raft_consensus.cc:2781] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:02.921476 471 raft_consensus.cc:686] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [term 1 LEADER]: Becoming Leader. State: Replica: cdd2a647857143ab8f752d2a17b697c4, State: Running, Role: LEADER
I20260619 16:54:02.921558 471 consensus_queue.cc:237] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } }
I20260619 16:54:02.921861 505 tablet_replica.cc:439] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: TxnStatusTablet state changed. Reason: New leader cdd2a647857143ab8f752d2a17b697c4. Latest consensus state: current_term: 1 leader_uuid: "cdd2a647857143ab8f752d2a17b697c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } }
I20260619 16:54:02.921959 505 tablet_replica.cc:442] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:54:02.922142 32424 catalog_manager.cc:5495] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdd2a647857143ab8f752d2a17b697c4 (127.30.148.131). New cstate: current_term: 1 leader_uuid: "cdd2a647857143ab8f752d2a17b697c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } health_report { overall_health: HEALTHY } } }
I20260619 16:54:02.922405 507 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 16:54:02.922508 507 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 16:54:02.932860 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.930086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:46191 (local address 127.0.0.1:53374)
0619 16:54:02.930229 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.930239 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.930283 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.930431 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.930431 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.930432 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:54:02.930468 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.930470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.931308 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.931309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.931891 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.931893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.932348 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.932349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.932425 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.932439 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:54:02.932715 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:54:02.932723 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.932773 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.932810 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260619 16:54:02.932861 459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.930165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.30.148.131:46191)
0619 16:54:02.930207 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.930209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.930272 (+ 63us) server_negotiation.cc:407] Connection header received
0619 16:54:02.930346 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.930346 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.930365 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.930431 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:02.930507 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.931229 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.931972 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.932269 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.932339 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.932532 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:02.932644 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:02.932717 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.932776 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.932795 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:54:02.933972 32723 raft_consensus.cc:1261] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b [term 1 FOLLOWER]: Refusing update from remote peer cdd2a647857143ab8f752d2a17b697c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:02.934063 32583 raft_consensus.cc:1261] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 1 FOLLOWER]: Refusing update from remote peer cdd2a647857143ab8f752d2a17b697c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:02.934252 505 consensus_queue.cc:1035] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:02.934374 471 consensus_queue.cc:1035] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:02.934815 470 tablet_replica.cc:439] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b: TxnStatusTablet state changed. Reason: New leader cdd2a647857143ab8f752d2a17b697c4. Latest consensus state: current_term: 1 leader_uuid: "cdd2a647857143ab8f752d2a17b697c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } }
I20260619 16:54:02.934904 470 tablet_replica.cc:442] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:02.935032 472 tablet_replica.cc:439] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: TxnStatusTablet state changed. Reason: New leader cdd2a647857143ab8f752d2a17b697c4. Latest consensus state: current_term: 1 leader_uuid: "cdd2a647857143ab8f752d2a17b697c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } }
I20260619 16:54:02.935127 472 tablet_replica.cc:442] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:02.935562 471 tablet_replica.cc:439] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cdd2a647857143ab8f752d2a17b697c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } }
I20260619 16:54:02.935636 471 tablet_replica.cc:442] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:54:02.935945 471 tablet_replica.cc:439] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cdd2a647857143ab8f752d2a17b697c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } }
I20260619 16:54:02.936012 471 tablet_replica.cc:442] T dbc089f7bc3b45e2a51ab07f797ac213 P cdd2a647857143ab8f752d2a17b697c4: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:54:02.936074 478 mvcc.cc:205] Tried to move back new op lower bound from 7298613423855685632 to 7298613423807127552. Current Snapshot: MvccSnapshot[applied={T|T < 7298613423855685632}]
I20260619 16:54:02.936154 470 tablet_replica.cc:439] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdd2a647857143ab8f752d2a17b697c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } }
I20260619 16:54:02.936208 470 tablet_replica.cc:442] T dbc089f7bc3b45e2a51ab07f797ac213 P f0729c76deb549ae997c759a0d50084b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:02.937213 477 mvcc.cc:205] Tried to move back new op lower bound from 7298613423855685632 to 7298613423807127552. Current Snapshot: MvccSnapshot[applied={T|T < 7298613423855685632 or (T in {7298613423855685632})}]
I20260619 16:54:02.937336 472 tablet_replica.cc:439] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdd2a647857143ab8f752d2a17b697c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 } } peers { permanent_uuid: "f0729c76deb549ae997c759a0d50084b" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 41813 } } peers { permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 } } }
I20260619 16:54:02.937428 472 tablet_replica.cc:442] T dbc089f7bc3b45e2a51ab07f797ac213 P 1bfce1c79e5a4056a5e1379cd6d1e6f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:02.939930 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.936996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59234 (local address 127.30.148.130:41813)
0619 16:54:02.937031 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.937033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.937410 (+ 377us) server_negotiation.cc:407] Connection header received
0619 16:54:02.937444 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.937445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.937467 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.937534 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:02.937693 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.938187 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.939235 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.939406 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.939456 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.939623 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:02.939712 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:02.939760 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.939831 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.939846 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 16:54:02.944402 459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.942153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:47089 (local address 127.30.148.131:46191)
0619 16:54:02.942194 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.942195 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.942300 (+ 105us) server_negotiation.cc:407] Connection header received
0619 16:54:02.942345 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.942346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.942354 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.942407 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:02.942529 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.942985 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.943743 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.943938 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.944010 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.944181 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:02.944216 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:02.944276 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.944328 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.944343 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:54:02.944446 514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.942081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:46191 (local address 127.30.148.130:47089)
0619 16:54:02.942219 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.942229 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.942292 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.942411 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.942412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.942413 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:54:02.942464 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.942466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.943065 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.943066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.943648 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.943649 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.944009 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.944010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.944073 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.944085 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:54:02.944276 (+ 191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:54:02.944278 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.944326 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.944361 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260619 16:54:02.947681 379 raft_consensus.cc:1261] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4 [term 1 FOLLOWER]: Refusing update from remote peer f0729c76deb549ae997c759a0d50084b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:02.947710 32583 raft_consensus.cc:1261] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 1 FOLLOWER]: Refusing update from remote peer f0729c76deb549ae997c759a0d50084b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:02.947991 493 consensus_queue.cc:1035] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdd2a647857143ab8f752d2a17b697c4" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:02.948168 493 consensus_queue.cc:1035] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bfce1c79e5a4056a5e1379cd6d1e6f5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:02.958084 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.955207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:41813 (local address 127.30.148.131:57853)
0619 16:54:02.955276 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:02.955287 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:02.955327 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:02.955481 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:02.955482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:02.955482 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:54:02.955553 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.955554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.956314 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.956315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.957252 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:02.957254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.957743 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.957744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:02.957796 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.957808 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:54:02.957976 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:54:02.957977 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 16:54:02.958006 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 16:54:02.958036 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 16:54:02.958101 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.955300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:57853 (local address 127.30.148.130:41813)
0619 16:54:02.955372 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.955374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.955381 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:54:02.955400 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.955400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.955410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.955467 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:02.955635 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.956237 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.957347 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.957671 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.957746 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.957885 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:02.957918 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:02.957977 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.958011 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.958032 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 16:54:02.967502 32425 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:02.967600 32425 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:02.968204 32425 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0bfa115380a4bb3a19bf88fde1850f0: Sending DeleteTablet for 3 replicas of tablet 03b6f0f0d64c4eb985945dae5165ea8b
I20260619 16:54:02.968431 32563 tablet_service.cc:1514] Processing DeleteTablet for tablet 03b6f0f0d64c4eb985945dae5165ea8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:02 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:51450
I20260619 16:54:02.968519 488 tablet_replica.cc:330] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5: stopping tablet replica
I20260619 16:54:02.968513 32703 tablet_service.cc:1514] Processing DeleteTablet for tablet 03b6f0f0d64c4eb985945dae5165ea8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:02 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:59224
I20260619 16:54:02.968569 488 raft_consensus.cc:2229] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:02.968588 487 tablet_replica.cc:330] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b: stopping tablet replica
I20260619 16:54:02.968600 488 raft_consensus.cc:2258] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:02.968633 487 raft_consensus.cc:2229] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:02.968709 487 raft_consensus.cc:2258] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:02.968899 375 tablet_service.cc:1514] Processing DeleteTablet for tablet 03b6f0f0d64c4eb985945dae5165ea8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:02 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:53358
I20260619 16:54:02.968982 486 tablet_replica.cc:330] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4: stopping tablet replica
I20260619 16:54:02.969022 486 raft_consensus.cc:2229] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:02.969053 486 raft_consensus.cc:2258] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:02.969100 488 ts_tablet_manager.cc:1905] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:02.969310 487 ts_tablet_manager.cc:1905] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:02.969327 486 ts_tablet_manager.cc:1905] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:02.970817 487 ts_tablet_manager.cc:1918] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:02.970835 486 ts_tablet_manager.cc:1918] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:02.970875 487 log.cc:1196] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/03b6f0f0d64c4eb985945dae5165ea8b
I20260619 16:54:02.970880 486 log.cc:1196] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/03b6f0f0d64c4eb985945dae5165ea8b
I20260619 16:54:02.970876 488 ts_tablet_manager.cc:1918] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:02.970925 488 log.cc:1196] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888036930978-31314-0/minicluster-data/ts-0/wal/wals/03b6f0f0d64c4eb985945dae5165ea8b
I20260619 16:54:02.971112 487 ts_tablet_manager.cc:1939] T 03b6f0f0d64c4eb985945dae5165ea8b P f0729c76deb549ae997c759a0d50084b: Deleting consensus metadata
I20260619 16:54:02.971112 486 ts_tablet_manager.cc:1939] T 03b6f0f0d64c4eb985945dae5165ea8b P cdd2a647857143ab8f752d2a17b697c4: Deleting consensus metadata
I20260619 16:54:02.971174 488 ts_tablet_manager.cc:1939] T 03b6f0f0d64c4eb985945dae5165ea8b P 1bfce1c79e5a4056a5e1379cd6d1e6f5: Deleting consensus metadata
I20260619 16:54:02.971503 32413 catalog_manager.cc:4841] TS f0729c76deb549ae997c759a0d50084b (127.30.148.130:41813): tablet 03b6f0f0d64c4eb985945dae5165ea8b (table test-table [id=c605b9fa21f241f5bf9aad941c331194]) successfully deleted
I20260619 16:54:02.971572 32413 catalog_manager.cc:4841] TS 1bfce1c79e5a4056a5e1379cd6d1e6f5 (127.30.148.129:32981): tablet 03b6f0f0d64c4eb985945dae5165ea8b (table test-table [id=c605b9fa21f241f5bf9aad941c331194]) successfully deleted
I20260619 16:54:02.971621 32413 catalog_manager.cc:4841] TS cdd2a647857143ab8f752d2a17b697c4 (127.30.148.131:46191): tablet 03b6f0f0d64c4eb985945dae5165ea8b (table test-table [id=c605b9fa21f241f5bf9aad941c331194]) successfully deleted
Jun 19 16:54:02 dist-test-slave-xb9n krb5kdc[32381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.148.129@KRBTEST.COM
I20260619 16:54:02.982386 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.972389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51476 (local address 127.30.148.129:32981)
0619 16:54:02.972427 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.972429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.972535 (+ 106us) server_negotiation.cc:407] Connection header received
0619 16:54:02.972593 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.972594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.972603 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.972663 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:02.973348 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.974115 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.975135 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.975439 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.975509 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.979025 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:02.979033 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:02.979034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:02.979042 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:02.981059 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.981503 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.981507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.981507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.981574 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.981821 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.981822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.981823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.981990 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:02.982132 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.982217 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.982296 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 16:54:02.982805 32623 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51476
I20260619 16:54:02.992959 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:02.985639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.30.148.190:33707)
0619 16:54:02.985704 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:02.985707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:02.985795 (+ 88us) server_negotiation.cc:407] Connection header received
0619 16:54:02.985842 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:02.985843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:02.985854 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:02.985913 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:02.986511 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.987270 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.988336 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:02.988680 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:02.988746 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:02.990142 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:02.990144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:02.990144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:02.990151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:02.991785 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.992156 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.992157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.992157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.992213 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:02.992427 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:02.992428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:02.992428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:02.992549 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:02.992665 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:02.992792 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 16:54:02.992866 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 16:54:02.993418 32425 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33930
I20260619 16:54:02.994076 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 32498
I20260619 16:54:03.002055 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 32638
I20260619 16:54:03.006929 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 310
I20260619 16:54:03.012069 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 32394
2026-06-19T16:54:03Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2207 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:03 dist-test-slave-xb9n krb5kdc[529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:03 dist-test-slave-xb9n krb5kdc[529](info): set up 2 sockets
Jun 19 16:54:03 dist-test-slave-xb9n krb5kdc[529](info): commencing operation
krb5kdc: starting...
W20260619 16:54:04.052896 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:04 dist-test-slave-xb9n krb5kdc[529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:54:04.163856 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:41625
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:34791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:41625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:04.242388 542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:04.242573 542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:04.242594 542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:04.243717 542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:04.243763 542 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 16:54:04.243777 542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:04.243790 542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:04.243803 542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:04.243857 542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:04.245270 542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:34791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:41625
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:41625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:04.245538 542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:04.245774 542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:04.250495 547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:04.250563 542 server_base.cc:1034] running on GCE node
W20260619 16:54:04.250524 550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:04.250499 548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:04.250939 542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:04.251173 542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:04.252313 542 hybrid_clock.cc:648] HybridClock initialized: now 1781888044252302 us; error 27 us; skew 500 ppm
Jun 19 16:54:04 dist-test-slave-xb9n krb5kdc[529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888044, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:04.254730 542 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:54:04.255137 542 webserver.cc:460] Webserver started at http://127.30.148.190:35809/ using document root <none> and password file <none>
I20260619 16:54:04.255347 542 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:04.255388 542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:04.255532 542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:04.256557 542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "63c6fa5d0f574a169f036c3f41f2d9d8"
format_stamp: "Formatted at 2026-06-19 16:54:04 on dist-test-slave-xb9n"
server_key: "000618cd7d2240aea8b26c11fbef6531"
server_key_iv: "dc4fc9cf8cd897d054608ea0c5b6559e"
server_key_version: "encryptionkey@0"
I20260619 16:54:04.256908 542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "63c6fa5d0f574a169f036c3f41f2d9d8"
format_stamp: "Formatted at 2026-06-19 16:54:04 on dist-test-slave-xb9n"
server_key: "000618cd7d2240aea8b26c11fbef6531"
server_key_iv: "dc4fc9cf8cd897d054608ea0c5b6559e"
server_key_version: "encryptionkey@0"
I20260619 16:54:04.258275 542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:54:04.258976 557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:04.259166 542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:04.259238 542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "63c6fa5d0f574a169f036c3f41f2d9d8"
format_stamp: "Formatted at 2026-06-19 16:54:04 on dist-test-slave-xb9n"
server_key: "000618cd7d2240aea8b26c11fbef6531"
server_key_iv: "dc4fc9cf8cd897d054608ea0c5b6559e"
server_key_version: "encryptionkey@0"
I20260619 16:54:04.259317 542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:04.290217 542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:04.292851 542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:04.293000 542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:04.297513 542 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:41625
I20260619 16:54:04.297559 619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:41625 every 8 connection(s)
I20260619 16:54:04.297899 542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:04.298110 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 542
I20260619 16:54:04.298206 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:04.298377 31314 external_mini_cluster.cc:1428] Setting key 2a2c32e757086a848298463bd1c54f1b
I20260619 16:54:04.298631 620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:04.301246 620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8: Bootstrap starting.
I20260619 16:54:04.301844 620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:04.302140 620 log.cc:824] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:04.302784 620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8: No bootstrap required, opened a new log
Jun 19 16:54:04 dist-test-slave-xb9n krb5kdc[529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:04.304517 620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c6fa5d0f574a169f036c3f41f2d9d8" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41625 } }
I20260619 16:54:04.304718 620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:04.304788 620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63c6fa5d0f574a169f036c3f41f2d9d8, State: Initialized, Role: FOLLOWER
I20260619 16:54:04.304936 620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c6fa5d0f574a169f036c3f41f2d9d8" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41625 } }
I20260619 16:54:04.305027 620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:04.305065 620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:04.305112 620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:04.305691 620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c6fa5d0f574a169f036c3f41f2d9d8" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41625 } }
I20260619 16:54:04.305805 620 leader_election.cc:302] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63c6fa5d0f574a169f036c3f41f2d9d8; no voters:
I20260619 16:54:04.305964 620 leader_election.cc:288] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:04.306123 625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:04.306247 620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:04.306267 625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [term 1 LEADER]: Becoming Leader. State: Replica: 63c6fa5d0f574a169f036c3f41f2d9d8, State: Running, Role: LEADER
I20260619 16:54:04.306348 625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c6fa5d0f574a169f036c3f41f2d9d8" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41625 } }
I20260619 16:54:04.306639 625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63c6fa5d0f574a169f036c3f41f2d9d8. Latest consensus state: current_term: 1 leader_uuid: "63c6fa5d0f574a169f036c3f41f2d9d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c6fa5d0f574a169f036c3f41f2d9d8" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41625 } } }
I20260619 16:54:04.306725 625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:04.306644 626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63c6fa5d0f574a169f036c3f41f2d9d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c6fa5d0f574a169f036c3f41f2d9d8" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41625 } } }
I20260619 16:54:04.306908 626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8 [sys.catalog]: This master's current role is: LEADER
W20260619 16:54:04.307971 640 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:54:04.308035 640 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:54:04.308106 633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:04.308259 633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:04.308724 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.298883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44048 (local address 127.30.148.190:41625)
0619 16:54:04.299080 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:04.299099 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:04.299123 (+ 24us) server_negotiation.cc:407] Connection header received
0619 16:54:04.299629 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:04.299635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:04.299681 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:04.299826 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:04.300463 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.301422 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.302266 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.302666 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.302795 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.305058 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:04.305061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:04.305062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:04.305079 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:04.306989 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.307373 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.307377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.307377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.307489 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.307818 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.307819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.307819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.308157 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:04.308308 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:04.308416 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 16:54:04.308549 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260619 16:54:04.309752 633 catalog_manager.cc:1347] Generated new cluster ID: db409857c61a466b8a7404515f1ba041
I20260619 16:54:04.309808 633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:04.315863 633 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:04.316370 633 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:04.332708 633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8: Generated new TSK 0
I20260619 16:54:04.332899 633 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:54:04.359622 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41625
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:34791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:04.438776 646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:04.438966 646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:04.438989 646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:04.440117 646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:04.440158 646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:04.440179 646 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 16:54:04.440230 646 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:54:04.440258 646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:04.441763 646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:34791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.148.190:41625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:04.442008 646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:04.442207 646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:54:04.447233 646 server_base.cc:1034] running on GCE node
W20260619 16:54:04.447188 652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:04.447188 651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:04.447180 654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:04.447570 646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:04.447820 646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:04.448987 646 hybrid_clock.cc:648] HybridClock initialized: now 1781888044448972 us; error 27 us; skew 500 ppm
Jun 19 16:54:04 dist-test-slave-xb9n krb5kdc[529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888044, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:04.451455 646 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:54:04.451828 646 webserver.cc:460] Webserver started at http://127.30.148.129:44497/ using document root <none> and password file <none>
I20260619 16:54:04.452016 646 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:04.452057 646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:04.452149 646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:04.453101 646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "4b7ea2754a62449996c2ea9cdf2023cd"
format_stamp: "Formatted at 2026-06-19 16:54:04 on dist-test-slave-xb9n"
server_key: "a1d6f91d59f4a73bf41d75f3939460ca"
server_key_iv: "5cf5a487e822b55e46c5a9c391636ced"
server_key_version: "encryptionkey@0"
I20260619 16:54:04.453469 646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "4b7ea2754a62449996c2ea9cdf2023cd"
format_stamp: "Formatted at 2026-06-19 16:54:04 on dist-test-slave-xb9n"
server_key: "a1d6f91d59f4a73bf41d75f3939460ca"
server_key_iv: "5cf5a487e822b55e46c5a9c391636ced"
server_key_version: "encryptionkey@0"
I20260619 16:54:04.454684 646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:04.455435 661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:04.455600 646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:04.455685 646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "4b7ea2754a62449996c2ea9cdf2023cd"
format_stamp: "Formatted at 2026-06-19 16:54:04 on dist-test-slave-xb9n"
server_key: "a1d6f91d59f4a73bf41d75f3939460ca"
server_key_iv: "5cf5a487e822b55e46c5a9c391636ced"
server_key_version: "encryptionkey@0"
I20260619 16:54:04.455737 646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:04.464116 646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:04.465848 646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:04.465953 646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:04.466360 646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:04.466394 646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:04.466419 646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:04.466472 646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:04.473392 646 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:33177
I20260619 16:54:04.473475 776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:33177 every 8 connection(s)
Jun 19 16:54:04 dist-test-slave-xb9n krb5kdc[529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888044, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:04.473804 646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:04.474102 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 646
I20260619 16:54:04.474189 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:04.474383 31314 external_mini_cluster.cc:1428] Setting key 8bfcd33773de8d11de375fd9b9be4ae0
I20260619 16:54:04.477048 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.466852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:34753 (local address 127.30.148.190:41625)
0619 16:54:04.466920 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:04.466922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:04.467106 (+ 184us) server_negotiation.cc:407] Connection header received
0619 16:54:04.467617 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:04.467618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:04.467630 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:04.467695 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:04.468607 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.469441 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.471095 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.471390 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.471464 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.474242 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:04.474243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:04.474244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:04.474251 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:04.475676 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.476097 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.476098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.476099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.476154 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.476414 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.476415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.476416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.476527 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:04.476600 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:04.476879 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 16:54:04.476962 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 16:54:04.477226 670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.466834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41625 (local address 127.30.148.129:34753)
0619 16:54:04.467001 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:04.467039 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:04.467480 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:04.467803 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:04.467809 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:04.468204 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:04.468496 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.468501 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.469539 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.469540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.470986 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.470989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.471480 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.471481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.471634 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.472232 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:04.472244 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:04.474124 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:04.475769 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:04.475776 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:04.475778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:04.476000 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:04.476233 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:04.476234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:04.476234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:04.476340 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:04.476592 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:04.476598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:04.476742 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 16:54:04.476848 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 16:54:04.476997 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":104,"threads_started":1}
I20260619 16:54:04.477699 777 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41625
I20260619 16:54:04.477779 777 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:04.477946 777 heartbeater.cc:511] Master 127.30.148.190:41625 requested a full tablet report, sending...
I20260619 16:54:04.478473 573 ts_manager.cc:194] Registered new tserver with Master: 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177)
I20260619 16:54:04.479333 573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:34753
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
I20260619 16:54:04.484995 784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.478757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41625 (local address 127.30.148.129:36839)
0619 16:54:04.478913 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:04.478924 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:04.478993 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:04.479129 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:04.479129 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:04.479328 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:04.479407 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.479409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.480476 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.480477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.481341 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.481344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.481730 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.481731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.481812 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.482174 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:04.482180 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:04.482620 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:04.484116 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:04.484117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:04.484117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:04.484283 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:04.484450 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:04.484451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:04.484451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:04.484500 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:04.484717 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:04.484718 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:04.484793 (+ 75us) client_negotiation.cc:769] Sending connection context
0619 16:54:04.484857 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 16:54:04.484929 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260619 16:54:04.485045 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.478917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:36839 (local address 127.30.148.190:41625)
0619 16:54:04.478947 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:04.478949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:04.478977 (+ 28us) server_negotiation.cc:407] Connection header received
0619 16:54:04.479052 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:04.479053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:04.479061 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:04.479113 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:04.479526 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.480355 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.481472 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.481648 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.481705 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.482706 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:04.482707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:04.482707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:04.482712 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:04.484046 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.484347 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.484348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.484348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.484395 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.484553 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.484554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.484554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.484642 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:04.484729 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:04.484902 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 16:54:04.484968 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:54:04.497918 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41625
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:34791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:04.575766 786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:04.575990 786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:04.576037 786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:04.577204 786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:04.577270 786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:04.577303 786 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 16:54:04.577333 786 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:54:04.577389 786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:04.578876 786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:34791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.148.190:41625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:04.579147 786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:04.579367 786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:04.584203 794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:04.584203 791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:04.584224 792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:04.584244 786 server_base.cc:1034] running on GCE node
I20260619 16:54:04.584779 786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:04.585017 786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:04.586172 786 hybrid_clock.cc:648] HybridClock initialized: now 1781888044586166 us; error 36 us; skew 500 ppm
Jun 19 16:54:04 dist-test-slave-xb9n krb5kdc[529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888044, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:04.588604 786 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:54:04.588977 786 webserver.cc:460] Webserver started at http://127.30.148.130:44663/ using document root <none> and password file <none>
I20260619 16:54:04.589161 786 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:04.589200 786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:04.589291 786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:04.590293 786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "795d9d72a3ee4b98a8e2c986ce1d2168"
format_stamp: "Formatted at 2026-06-19 16:54:04 on dist-test-slave-xb9n"
server_key: "575befe98d5bbc9badbbe0aa6f76e512"
server_key_iv: "24db0961325298f58f362495ed53a230"
server_key_version: "encryptionkey@0"
I20260619 16:54:04.590579 786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "795d9d72a3ee4b98a8e2c986ce1d2168"
format_stamp: "Formatted at 2026-06-19 16:54:04 on dist-test-slave-xb9n"
server_key: "575befe98d5bbc9badbbe0aa6f76e512"
server_key_iv: "24db0961325298f58f362495ed53a230"
server_key_version: "encryptionkey@0"
I20260619 16:54:04.591773 786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:54:04.592398 801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:04.592555 786 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:04.592633 786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "795d9d72a3ee4b98a8e2c986ce1d2168"
format_stamp: "Formatted at 2026-06-19 16:54:04 on dist-test-slave-xb9n"
server_key: "575befe98d5bbc9badbbe0aa6f76e512"
server_key_iv: "24db0961325298f58f362495ed53a230"
server_key_version: "encryptionkey@0"
I20260619 16:54:04.592681 786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:04.600584 786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:04.602403 786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:04.602519 786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:04.602977 786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:04.603016 786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:04.603037 786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:04.603076 786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 16:54:04 dist-test-slave-xb9n krb5kdc[529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888044, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:04.610164 786 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:46643
I20260619 16:54:04.610232 916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:46643 every 8 connection(s)
I20260619 16:54:04.610502 786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:54:04.612286 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 786
I20260619 16:54:04.612382 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:04.612581 31314 external_mini_cluster.cc:1428] Setting key 7d71c5c3a77196b18791ca80455ccf38
I20260619 16:54:04.613510 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.603472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:41193 (local address 127.30.148.190:41625)
0619 16:54:04.603543 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:04.603546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:04.603693 (+ 147us) server_negotiation.cc:407] Connection header received
0619 16:54:04.604166 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:04.604167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:04.604181 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:04.604266 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:04.605075 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.605943 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.607622 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.607986 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.608065 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.610652 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:04.610653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:04.610653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:04.610661 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:04.611941 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.612388 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.612390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.612390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.612459 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.612823 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.612824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.612824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.612930 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:04.612985 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:04.613308 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 16:54:04.613414 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 16:54:04.613727 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.603404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41625 (local address 127.30.148.130:41193)
0619 16:54:04.603571 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:04.603609 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:04.604072 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:04.604348 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:04.604353 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:04.604681 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:04.604969 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.604975 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.606040 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.606041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.607519 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.607522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.608085 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.608086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.608245 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.608729 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:04.608740 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:04.610574 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:04.612027 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:04.612033 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:04.612035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:04.612261 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:04.612538 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:04.612539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:04.612539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:04.612670 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:04.612993 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:04.612998 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:04.613146 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 16:54:04.613283 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 16:54:04.613490 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":88,"threads_started":1}
I20260619 16:54:04.614195 917 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41625
I20260619 16:54:04.614275 917 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:04.614454 917 heartbeater.cc:511] Master 127.30.148.190:41625 requested a full tablet report, sending...
I20260619 16:54:04.614818 573 ts_manager.cc:194] Registered new tserver with Master: 795d9d72a3ee4b98a8e2c986ce1d2168 (127.30.148.130:46643)
I20260619 16:54:04.615228 573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:41193
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
I20260619 16:54:04.623224 924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.617595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41625 (local address 127.30.148.130:36211)
0619 16:54:04.617759 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:04.617768 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:04.617820 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:04.617962 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:04.617963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:04.618112 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:04.618164 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.618165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.619137 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.619138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.619621 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.619622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.620091 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.620091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.620141 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.620581 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:04.620585 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:04.620861 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:04.622372 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:04.622373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:04.622373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:04.622556 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:04.622709 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:04.622710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:04.622710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:04.622751 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:04.622954 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:04.622955 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:04.623001 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 16:54:04.623064 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 16:54:04.623133 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
I20260619 16:54:04.623266 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.617658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:36211 (local address 127.30.148.190:41625)
0619 16:54:04.617695 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:04.617697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:04.617861 (+ 164us) server_negotiation.cc:407] Connection header received
0619 16:54:04.617885 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:04.617886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:04.617898 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:04.617955 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:04.618242 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.619032 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.619700 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.620012 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.620089 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.620953 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:04.620954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:04.620955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:04.620962 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:04.622302 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.622620 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.622620 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.622621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.622671 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.622785 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.622785 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.622785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.622901 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:04.622950 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:04.623083 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 16:54:04.623141 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:54:04.637090 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41625
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:34791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:04.720041 926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:04.720245 926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:04.720270 926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:04.721398 926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:04.721453 926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:04.721477 926 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 16:54:04.721491 926 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:54:04.721509 926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:04.722975 926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:34791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.148.190:41625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:04.723207 926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:04.723426 926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:54:04.728283 926 server_base.cc:1034] running on GCE node
W20260619 16:54:04.728279 931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:04.728235 934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:04.728235 932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:04.728660 926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:04.728936 926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:04.730090 926 hybrid_clock.cc:648] HybridClock initialized: now 1781888044730077 us; error 26 us; skew 500 ppm
Jun 19 16:54:04 dist-test-slave-xb9n krb5kdc[529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888044, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:04.732470 926 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:54:04.732885 926 webserver.cc:460] Webserver started at http://127.30.148.131:33257/ using document root <none> and password file <none>
I20260619 16:54:04.733100 926 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:04.733144 926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:04.733289 926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:04.734308 926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "bb2f823b184a42188c93a89bb9133798"
format_stamp: "Formatted at 2026-06-19 16:54:04 on dist-test-slave-xb9n"
server_key: "850f0b6eaa3e04bff9f29620c07a7a4b"
server_key_iv: "e03449e32c6f1d97ff83b2a0a788408b"
server_key_version: "encryptionkey@0"
I20260619 16:54:04.734655 926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "bb2f823b184a42188c93a89bb9133798"
format_stamp: "Formatted at 2026-06-19 16:54:04 on dist-test-slave-xb9n"
server_key: "850f0b6eaa3e04bff9f29620c07a7a4b"
server_key_iv: "e03449e32c6f1d97ff83b2a0a788408b"
server_key_version: "encryptionkey@0"
I20260619 16:54:04.735918 926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:04.736622 941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:04.736805 926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:04.736892 926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "bb2f823b184a42188c93a89bb9133798"
format_stamp: "Formatted at 2026-06-19 16:54:04 on dist-test-slave-xb9n"
server_key: "850f0b6eaa3e04bff9f29620c07a7a4b"
server_key_iv: "e03449e32c6f1d97ff83b2a0a788408b"
server_key_version: "encryptionkey@0"
I20260619 16:54:04.736963 926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:04.757117 926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:04.759496 926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:04.759670 926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:04.760152 926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:04.760214 926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:04.760257 926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:04.760285 926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:04.766826 926 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:34971
I20260619 16:54:04.766892 1056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:34971 every 8 connection(s)
Jun 19 16:54:04 dist-test-slave-xb9n krb5kdc[529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888044, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:04.767189 926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:54:04.770061 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.760616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:52343 (local address 127.30.148.190:41625)
0619 16:54:04.760693 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:04.760696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:04.760868 (+ 172us) server_negotiation.cc:407] Connection header received
0619 16:54:04.761321 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:04.761322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:04.761336 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:04.761440 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:04.762282 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.763190 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.764768 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.765106 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.765190 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.767763 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:04.767764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:04.767765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:04.767771 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:04.768905 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.769266 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.769267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.769267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.769316 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.769572 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.769573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.769573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.769674 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:04.769725 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:04.769941 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 16:54:04.769998 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 16:54:04.770274 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.760588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41625 (local address 127.30.148.131:52343)
0619 16:54:04.760724 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:04.760775 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:04.761227 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:04.761522 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:04.761527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:04.761900 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:04.762179 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.762187 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.763306 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.763307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.764652 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.764654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.765196 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.765196 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.765304 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.765886 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:04.765897 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:04.767666 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:04.768986 (+ 1320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:04.768992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:04.768994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:04.769201 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:04.769411 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:04.769411 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:04.769412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:04.769516 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:04.769729 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:04.769738 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:04.769843 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 16:54:04.769922 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 16:54:04.770036 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260619 16:54:04.770789 1057 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41625
I20260619 16:54:04.770862 1057 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:04.771042 1057 heartbeater.cc:511] Master 127.30.148.190:41625 requested a full tablet report, sending...
I20260619 16:54:04.771469 573 ts_manager.cc:194] Registered new tserver with Master: bb2f823b184a42188c93a89bb9133798 (127.30.148.131:34971)
I20260619 16:54:04.771983 573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:52343
I20260619 16:54:04.772074 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 926
I20260619 16:54:04.772166 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:04.772387 31314 external_mini_cluster.cc:1428] Setting key af25214480142e95d3d8bc0aea505061
I20260619 16:54:04.773231 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:54:04.777674 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.771843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:33745 (local address 127.30.148.190:41625)
0619 16:54:04.771882 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:04.771884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:04.771966 (+ 82us) server_negotiation.cc:407] Connection header received
0619 16:54:04.772060 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:04.772060 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:04.772070 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:04.772110 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:04.772576 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.773199 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.773944 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.774146 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.774227 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.775160 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:04.775162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:04.775162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:04.775168 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:04.776540 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.776903 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.776904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.776905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.776956 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.777151 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.777152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.777152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.777260 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:04.777336 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:04.777517 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 16:54:04.777587 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:54:04.777693 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.771762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41625 (local address 127.30.148.131:33745)
0619 16:54:04.771884 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:04.771894 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:04.771990 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:04.772183 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:04.772184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:04.772381 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:04.772471 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.772472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.773290 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.773292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.773846 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.773847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.774229 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.774230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.774305 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.774749 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:04.774754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:04.775048 (+ 294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:04.776646 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:04.776647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:04.776648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:04.776819 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:04.777027 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:04.777028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:04.777028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:04.777084 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:04.777327 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:04.777328 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:04.777402 (+ 74us) client_negotiation.cc:769] Sending connection context
0619 16:54:04.777490 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 16:54:04.777597 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
Jun 19 16:54:04 dist-test-slave-xb9n krb5kdc[529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 16:54:04 dist-test-slave-xb9n krb5kdc[529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:04.795185 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.788870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44064 (local address 127.30.148.190:41625)
0619 16:54:04.788936 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:04.788938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:04.788948 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:04.788973 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:04.788974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:04.788981 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:04.789026 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:04.789577 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.790038 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.790718 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.790883 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.790930 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.793028 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:04.793029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:04.793029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:04.793035 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:04.794194 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.794484 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.794485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.794485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.794530 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:04.794717 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:04.794717 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:04.794717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:04.794808 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:04.794877 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:04.795029 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 16:54:04.795080 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 16:54:04.796713 573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:04.798309 573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:04.804570 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.800368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:46643 (local address 127.0.0.1:56038)
0619 16:54:04.800664 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:04.800706 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:04.800795 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:04.801253 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:04.801254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:04.801257 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:04.801293 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.801294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.802493 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.802494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.803292 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.803294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.804173 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.804174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.804270 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.804284 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 16:54:04.804373 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 16:54:04.804431 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":51,"threads_started":1}
I20260619 16:54:04.804571 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.800662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.30.148.130:46643)
0619 16:54:04.801008 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:04.801012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:04.801020 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:04.801085 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:04.801087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:04.801103 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:04.801177 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:04.801389 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.802410 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.803383 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.804083 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.804166 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.804237 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:04.804398 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 16:54:04.804445 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":84,"threads_started":1}
I20260619 16:54:04.804734 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.800661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32896 (local address 127.30.148.129:33177)
0619 16:54:04.800840 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:04.800845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:04.800954 (+ 109us) server_negotiation.cc:407] Connection header received
0619 16:54:04.801012 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:04.801014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:04.801027 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:04.801234 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:04.801426 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.802086 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.803456 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.804105 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.804166 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.804248 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:04.804568 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 16:54:04.804620 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":100,"threads_started":1}
I20260619 16:54:04.804809 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.800368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:34971 (local address 127.0.0.1:42514)
0619 16:54:04.800841 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:04.800850 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:04.800892 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:04.801120 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:04.801124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:04.801127 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:04.801187 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.801189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.802463 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.802464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.803711 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.803713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.804568 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.804569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.804631 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.804635 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:04.804690 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 16:54:04.804728 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":63,"thread_start_us":52,"threads_started":1}
I20260619 16:54:04.804811 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.800548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:33177 (local address 127.0.0.1:32896)
0619 16:54:04.800893 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:04.800901 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:04.800946 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:04.801296 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:04.801297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:04.801299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:04.801346 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.801348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.802181 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.802182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.803390 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:04.803392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.804281 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.804282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:04.804377 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.804383 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 16:54:04.804434 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 16:54:04.804490 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":18,"thread_start_us":58,"threads_started":1}
I20260619 16:54:04.805054 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:04.800488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.30.148.131:34971)
0619 16:54:04.800881 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:04.800885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:04.800895 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:04.800985 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:04.800987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:04.801014 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:04.801101 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:04.801268 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.802377 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.803789 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:04.804372 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:04.804431 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:04.804497 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:04.804910 (+ 413us) server_negotiation.cc:299] Negotiation successful
0619 16:54:04.804962 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":110,"threads_started":1}
I20260619 16:54:04.805548 711 tablet_service.cc:1467] Processing CreateTablet for tablet 5c86c3ac7b81493584ca177076eb324d (DEFAULT_TABLE table=test-table [id=f1f51233d3cb4f7fae3fd521f337c95f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:04.805675 991 tablet_service.cc:1467] Processing CreateTablet for tablet 5c86c3ac7b81493584ca177076eb324d (DEFAULT_TABLE table=test-table [id=f1f51233d3cb4f7fae3fd521f337c95f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:04.805953 711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c86c3ac7b81493584ca177076eb324d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:04.805974 991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c86c3ac7b81493584ca177076eb324d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:04.809203 1077 tablet_bootstrap.cc:492] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798: Bootstrap starting.
I20260619 16:54:04.809592 1078 tablet_bootstrap.cc:492] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd: Bootstrap starting.
I20260619 16:54:04.809616 851 tablet_service.cc:1467] Processing CreateTablet for tablet 5c86c3ac7b81493584ca177076eb324d (DEFAULT_TABLE table=test-table [id=f1f51233d3cb4f7fae3fd521f337c95f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:04.809990 851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c86c3ac7b81493584ca177076eb324d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:04.810230 1077 tablet_bootstrap.cc:654] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:04.810246 1078 tablet_bootstrap.cc:654] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:04.810586 1077 log.cc:824] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:04.810586 1078 log.cc:824] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:04.812826 1081 tablet_bootstrap.cc:492] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168: Bootstrap starting.
I20260619 16:54:04.813351 1078 tablet_bootstrap.cc:492] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd: No bootstrap required, opened a new log
I20260619 16:54:04.813453 1077 tablet_bootstrap.cc:492] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798: No bootstrap required, opened a new log
I20260619 16:54:04.813449 1078 ts_tablet_manager.cc:1397] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 16:54:04.813566 1077 ts_tablet_manager.cc:1397] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 16:54:04.815454 1078 raft_consensus.cc:348] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } }
I20260619 16:54:04.815454 1077 raft_consensus.cc:348] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } }
I20260619 16:54:04.815619 1078 raft_consensus.cc:374] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:04.815619 1077 raft_consensus.cc:374] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:04.815654 1078 raft_consensus.cc:729] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b7ea2754a62449996c2ea9cdf2023cd, State: Initialized, Role: FOLLOWER
I20260619 16:54:04.815654 1077 raft_consensus.cc:729] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2f823b184a42188c93a89bb9133798, State: Initialized, Role: FOLLOWER
I20260619 16:54:04.815768 1078 consensus_queue.cc:260] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } }
I20260619 16:54:04.815768 1077 consensus_queue.cc:260] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } }
I20260619 16:54:04.816030 1078 ts_tablet_manager.cc:1428] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:54:04.816025 1077 ts_tablet_manager.cc:1428] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260619 16:54:04.816149 1057 heartbeater.cc:503] Master 127.30.148.190:41625 was elected leader, sending a full tablet report...
I20260619 16:54:04.816160 777 heartbeater.cc:503] Master 127.30.148.190:41625 was elected leader, sending a full tablet report...
I20260619 16:54:04.816454 1081 tablet_bootstrap.cc:654] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:04.816788 1081 log.cc:824] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:04.817693 1081 tablet_bootstrap.cc:492] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168: No bootstrap required, opened a new log
I20260619 16:54:04.817744 1081 ts_tablet_manager.cc:1397] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 16:54:04.818940 1081 raft_consensus.cc:348] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } }
I20260619 16:54:04.819058 1081 raft_consensus.cc:374] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:04.819079 1081 raft_consensus.cc:729] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 795d9d72a3ee4b98a8e2c986ce1d2168, State: Initialized, Role: FOLLOWER
I20260619 16:54:04.819180 1081 consensus_queue.cc:260] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } }
I20260619 16:54:04.819380 1081 ts_tablet_manager.cc:1428] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:54:04.819461 917 heartbeater.cc:503] Master 127.30.148.190:41625 was elected leader, sending a full tablet report...
W20260619 16:54:04.861181 918 tablet.cc:2369] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:54:04.975099 778 tablet.cc:2369] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:54:05.017799 1058 tablet.cc:2369] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:54:05.129781 1082 raft_consensus.cc:482] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:05.129895 1082 raft_consensus.cc:504] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } }
I20260619 16:54:05.130191 1082 leader_election.cc:288] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 795d9d72a3ee4b98a8e2c986ce1d2168 (127.30.148.130:46643), 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177)
I20260619 16:54:05.133194 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.130611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:41017 (local address 127.30.148.129:33177)
0619 16:54:05.130654 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:05.130656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:05.130760 (+ 104us) server_negotiation.cc:407] Connection header received
0619 16:54:05.130805 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:05.130806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:05.130816 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:05.130866 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:05.131014 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.131514 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.132444 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.132843 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.132898 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.132905 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:05.133075 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 16:54:05.133121 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:54:05.133234 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.130445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:41389 (local address 127.30.148.130:46643)
0619 16:54:05.130485 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:05.130487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:05.130520 (+ 33us) server_negotiation.cc:407] Connection header received
0619 16:54:05.130559 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:05.130560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:05.130570 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:05.130645 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:05.130883 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.131428 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.132541 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.132868 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.132961 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.132969 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:05.133135 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 16:54:05.133172 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:54:05.133214 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.130519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:33177 (local address 127.30.148.131:41017)
0619 16:54:05.130681 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:05.130691 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:05.130752 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:05.130880 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:05.130881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:05.130882 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:05.130942 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.130944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.131602 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.131602 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.132365 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.132366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.132921 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.132922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.133001 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.133006 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:54:05.133061 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 16:54:05.133088 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260619 16:54:05.133668 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.130397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:46643 (local address 127.30.148.131:41389)
0619 16:54:05.130449 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:05.130461 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:05.130506 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:05.130710 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:05.130711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:05.130713 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:05.130806 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.130808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.131534 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.131535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.132459 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.132460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.132971 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.132971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.133057 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.133061 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:05.133117 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 16:54:05.133152 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":37}
I20260619 16:54:05.133628 731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c86c3ac7b81493584ca177076eb324d" candidate_uuid: "bb2f823b184a42188c93a89bb9133798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" is_pre_election: true
I20260619 16:54:05.133836 731 raft_consensus.cc:2454] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb2f823b184a42188c93a89bb9133798 in term 0.
I20260619 16:54:05.134024 871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c86c3ac7b81493584ca177076eb324d" candidate_uuid: "bb2f823b184a42188c93a89bb9133798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" is_pre_election: true
I20260619 16:54:05.134135 871 raft_consensus.cc:2454] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb2f823b184a42188c93a89bb9133798 in term 0.
I20260619 16:54:05.134075 943 leader_election.cc:302] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b7ea2754a62449996c2ea9cdf2023cd, bb2f823b184a42188c93a89bb9133798; no voters:
I20260619 16:54:05.134251 1082 raft_consensus.cc:2781] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:05.134307 1082 raft_consensus.cc:482] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:05.134330 1082 raft_consensus.cc:3037] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:05.135004 1082 raft_consensus.cc:504] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } }
I20260619 16:54:05.135131 1082 leader_election.cc:288] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [CANDIDATE]: Term 1 election: Requested vote from peers 795d9d72a3ee4b98a8e2c986ce1d2168 (127.30.148.130:46643), 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177)
I20260619 16:54:05.135263 871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c86c3ac7b81493584ca177076eb324d" candidate_uuid: "bb2f823b184a42188c93a89bb9133798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168"
I20260619 16:54:05.135289 731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c86c3ac7b81493584ca177076eb324d" candidate_uuid: "bb2f823b184a42188c93a89bb9133798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7ea2754a62449996c2ea9cdf2023cd"
I20260619 16:54:05.135339 871 raft_consensus.cc:3037] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:05.135402 731 raft_consensus.cc:3037] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:05.136026 871 raft_consensus.cc:2454] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb2f823b184a42188c93a89bb9133798 in term 1.
I20260619 16:54:05.136178 731 raft_consensus.cc:2454] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb2f823b184a42188c93a89bb9133798 in term 1.
I20260619 16:54:05.136255 943 leader_election.cc:302] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 795d9d72a3ee4b98a8e2c986ce1d2168, bb2f823b184a42188c93a89bb9133798; no voters:
I20260619 16:54:05.136368 1082 raft_consensus.cc:2781] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:05.136504 1082 raft_consensus.cc:686] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 1 LEADER]: Becoming Leader. State: Replica: bb2f823b184a42188c93a89bb9133798, State: Running, Role: LEADER
I20260619 16:54:05.136616 1082 consensus_queue.cc:237] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } }
I20260619 16:54:05.137446 573 catalog_manager.cc:5495] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb2f823b184a42188c93a89bb9133798 (127.30.148.131). New cstate: current_term: 1 leader_uuid: "bb2f823b184a42188c93a89bb9133798" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } health_report { overall_health: UNKNOWN } } }
Jun 19 16:54:05 dist-test-slave-xb9n krb5kdc[529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888044, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:05.172744 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.165936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41625 (local address 127.0.0.1:44080)
0619 16:54:05.166131 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:05.166143 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:05.166224 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:05.166373 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:05.166374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:05.166700 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:05.166752 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.166754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.167395 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.167396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.167942 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.167944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.168318 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.168319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.168389 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.168938 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:05.168942 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:05.170260 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:05.171830 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:05.171836 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:05.171838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:05.172040 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:05.172209 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:05.172209 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:05.172209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:05.172266 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:05.172465 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:05.172470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:05.172513 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 16:54:05.172568 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 16:54:05.172653 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":120,"threads_started":1}
I20260619 16:54:05.172796 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.166019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.30.148.190:41625)
0619 16:54:05.166103 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:05.166105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:05.166211 (+ 106us) server_negotiation.cc:407] Connection header received
0619 16:54:05.166277 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:05.166278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:05.166288 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:05.166367 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:05.166835 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.167313 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.168035 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.168213 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.168298 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.170350 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:05.170352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:05.170352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:05.170357 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:05.171759 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:05.172104 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:05.172105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:05.172105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:05.172153 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:05.172318 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:05.172319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:05.172319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:05.172412 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:05.172462 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:05.172598 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 16:54:05.172696 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":56}
I20260619 16:54:05.174866 573 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:44080:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 16:54:05.175105 573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:05.176509 711 tablet_service.cc:1467] Processing CreateTablet for tablet af941d11fef24f56b471df3402e8b310 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=52f2c674762a404a8ecae95c6a473fec]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:54:05.176542 851 tablet_service.cc:1467] Processing CreateTablet for tablet af941d11fef24f56b471df3402e8b310 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=52f2c674762a404a8ecae95c6a473fec]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:54:05.176673 851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af941d11fef24f56b471df3402e8b310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:05.176692 711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af941d11fef24f56b471df3402e8b310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:05.176752 991 tablet_service.cc:1467] Processing CreateTablet for tablet af941d11fef24f56b471df3402e8b310 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=52f2c674762a404a8ecae95c6a473fec]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:54:05.176892 991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af941d11fef24f56b471df3402e8b310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:05.177891 1081 tablet_bootstrap.cc:492] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: Bootstrap starting.
I20260619 16:54:05.178275 1078 tablet_bootstrap.cc:492] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd: Bootstrap starting.
I20260619 16:54:05.178340 1077 tablet_bootstrap.cc:492] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798: Bootstrap starting.
I20260619 16:54:05.178360 1081 tablet_bootstrap.cc:654] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:05.178650 1078 tablet_bootstrap.cc:654] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:05.178761 1077 tablet_bootstrap.cc:654] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:05.179239 1078 tablet_bootstrap.cc:492] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd: No bootstrap required, opened a new log
I20260619 16:54:05.179304 1078 ts_tablet_manager.cc:1397] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:05.179426 1081 tablet_bootstrap.cc:492] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: No bootstrap required, opened a new log
I20260619 16:54:05.179474 1081 ts_tablet_manager.cc:1397] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:05.179481 1078 raft_consensus.cc:348] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.179566 1078 raft_consensus.cc:374] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:05.179592 1078 raft_consensus.cc:729] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b7ea2754a62449996c2ea9cdf2023cd, State: Initialized, Role: FOLLOWER
I20260619 16:54:05.179651 1077 tablet_bootstrap.cc:492] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798: No bootstrap required, opened a new log
I20260619 16:54:05.179641 1078 consensus_queue.cc:260] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.179617 1081 raft_consensus.cc:348] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.179689 1077 ts_tablet_manager.cc:1397] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 16:54:05.179701 1081 raft_consensus.cc:374] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:05.179726 1081 raft_consensus.cc:729] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 795d9d72a3ee4b98a8e2c986ce1d2168, State: Initialized, Role: FOLLOWER
I20260619 16:54:05.179754 1078 ts_tablet_manager.cc:1428] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:05.179780 1081 consensus_queue.cc:260] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.179797 1083 tablet_replica.cc:439] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } }
I20260619 16:54:05.179826 1077 raft_consensus.cc:348] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.179890 1077 raft_consensus.cc:374] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:05.179883 1081 ts_tablet_manager.cc:1428] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:05.179877 1083 tablet_replica.cc:442] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:05.179904 1077 raft_consensus.cc:729] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2f823b184a42188c93a89bb9133798, State: Initialized, Role: FOLLOWER
I20260619 16:54:05.179960 1077 consensus_queue.cc:260] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.180061 1077 ts_tablet_manager.cc:1428] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:05.180114 1085 tablet_replica.cc:439] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } }
I20260619 16:54:05.180151 1082 tablet_replica.cc:439] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } }
I20260619 16:54:05.180197 1085 tablet_replica.cc:442] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:05.180228 1082 tablet_replica.cc:442] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:05.617496 1085 raft_consensus.cc:482] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:05.617578 1085 raft_consensus.cc:504] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.617909 1085 leader_election.cc:288] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177), bb2f823b184a42188c93a89bb9133798 (127.30.148.131:34971)
I20260619 16:54:05.621963 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.618168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:48665 (local address 127.30.148.129:33177)
0619 16:54:05.618214 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:05.618216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:05.618272 (+ 56us) server_negotiation.cc:407] Connection header received
0619 16:54:05.618377 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:05.618378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:05.618389 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:05.618445 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:05.618645 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.619410 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.621022 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.621597 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.621676 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.621685 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:05.621855 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 16:54:05.621894 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:54:05.621973 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.618177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:44873 (local address 127.30.148.131:34971)
0619 16:54:05.618283 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:05.618285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:05.618301 (+ 16us) server_negotiation.cc:407] Connection header received
0619 16:54:05.618342 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:05.618343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:05.618355 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:05.618445 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:05.618645 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.619403 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.621022 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.621608 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.621677 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.621685 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:05.621855 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 16:54:05.621893 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":65,"threads_started":1}
I20260619 16:54:05.621995 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.618090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:34971 (local address 127.30.148.130:44873)
0619 16:54:05.618203 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:05.618216 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:05.618269 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:05.618467 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:05.618468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:05.618470 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:05.618547 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.618550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.619495 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.619495 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.620923 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.620926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.621689 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.621691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.621764 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.621769 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:54:05.621829 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 16:54:05.621885 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":102,"thread_start_us":56,"threads_started":1}
I20260619 16:54:05.622004 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.618090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:33177 (local address 127.30.148.130:48665)
0619 16:54:05.618204 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:05.618216 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:05.618297 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:05.618467 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:05.618468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:05.618472 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:05.618569 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.618571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.619494 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.619495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.620923 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.620926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.621691 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.621691 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.621764 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.621770 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 16:54:05.621829 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 16:54:05.621885 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":142,"thread_start_us":46,"threads_started":1}
I20260619 16:54:05.622303 1011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af941d11fef24f56b471df3402e8b310" candidate_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2f823b184a42188c93a89bb9133798" is_pre_election: true
I20260619 16:54:05.622429 1011 raft_consensus.cc:2454] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 795d9d72a3ee4b98a8e2c986ce1d2168 in term 0.
I20260619 16:54:05.622525 731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af941d11fef24f56b471df3402e8b310" candidate_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" is_pre_election: true
I20260619 16:54:05.622614 731 raft_consensus.cc:2454] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 795d9d72a3ee4b98a8e2c986ce1d2168 in term 0.
I20260619 16:54:05.622645 802 leader_election.cc:302] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 795d9d72a3ee4b98a8e2c986ce1d2168, bb2f823b184a42188c93a89bb9133798; no voters:
I20260619 16:54:05.622792 1085 raft_consensus.cc:2781] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:05.622852 1085 raft_consensus.cc:482] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:05.622884 1085 raft_consensus.cc:3037] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:05.623528 1085 raft_consensus.cc:504] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.623667 1085 leader_election.cc:288] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [CANDIDATE]: Term 1 election: Requested vote from peers 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177), bb2f823b184a42188c93a89bb9133798 (127.30.148.131:34971)
I20260619 16:54:05.623808 731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af941d11fef24f56b471df3402e8b310" candidate_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7ea2754a62449996c2ea9cdf2023cd"
I20260619 16:54:05.623883 731 raft_consensus.cc:3037] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:05.624410 731 raft_consensus.cc:2454] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 795d9d72a3ee4b98a8e2c986ce1d2168 in term 1.
I20260619 16:54:05.624529 1011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af941d11fef24f56b471df3402e8b310" candidate_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2f823b184a42188c93a89bb9133798"
I20260619 16:54:05.624604 1011 raft_consensus.cc:3037] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:05.624590 803 leader_election.cc:302] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b7ea2754a62449996c2ea9cdf2023cd, 795d9d72a3ee4b98a8e2c986ce1d2168; no voters:
I20260619 16:54:05.625200 1011 raft_consensus.cc:2454] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 795d9d72a3ee4b98a8e2c986ce1d2168 in term 1.
I20260619 16:54:05.625309 1085 raft_consensus.cc:2781] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:05.625468 1085 raft_consensus.cc:686] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 LEADER]: Becoming Leader. State: Replica: 795d9d72a3ee4b98a8e2c986ce1d2168, State: Running, Role: LEADER
I20260619 16:54:05.625576 1085 consensus_queue.cc:237] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.626026 1101 tablet_replica.cc:439] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: TxnStatusTablet state changed. Reason: New leader 795d9d72a3ee4b98a8e2c986ce1d2168. Latest consensus state: current_term: 1 leader_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } }
I20260619 16:54:05.626118 1101 tablet_replica.cc:442] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:54:05.626338 1103 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 16:54:05.626446 1103 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 16:54:05.626384 574 catalog_manager.cc:5495] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 795d9d72a3ee4b98a8e2c986ce1d2168 (127.30.148.130). New cstate: current_term: 1 leader_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } health_report { overall_health: UNKNOWN } } }
I20260619 16:54:05.670367 1082 consensus_queue.cc:1035] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:05.672606 1109 consensus_queue.cc:1035] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:05.730595 1114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.727885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:46643 (local address 127.0.0.1:56042)
0619 16:54:05.727998 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:05.728010 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:05.728072 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:05.728275 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:05.728276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:05.728278 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:54:05.728332 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.728334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.729012 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.729013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.729684 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.729686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.730070 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.730071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.730129 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.730141 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:54:05.730406 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:54:05.730415 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 16:54:05.730463 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 16:54:05.730512 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":49,"threads_started":1}
I20260619 16:54:05.730582 1115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.727969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56042 (local address 127.30.148.130:46643)
0619 16:54:05.728081 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:05.728083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:05.728092 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:54:05.728151 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:05.728152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:05.728170 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:05.728255 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:05.728409 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.728930 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.729768 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.729995 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.730072 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.730243 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:05.730338 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:05.730426 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:05.730470 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 16:54:05.730492 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260619 16:54:05.732367 731 raft_consensus.cc:1261] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Refusing update from remote peer 795d9d72a3ee4b98a8e2c986ce1d2168: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:05.732426 1011 raft_consensus.cc:1261] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Refusing update from remote peer 795d9d72a3ee4b98a8e2c986ce1d2168: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:05.732702 1101 consensus_queue.cc:1035] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:05.732798 1085 consensus_queue.cc:1035] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:05.733700 1083 tablet_replica.cc:439] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd: TxnStatusTablet state changed. Reason: New leader 795d9d72a3ee4b98a8e2c986ce1d2168. Latest consensus state: current_term: 1 leader_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } }
I20260619 16:54:05.733793 1083 tablet_replica.cc:442] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:05.733845 1109 tablet_replica.cc:439] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798: TxnStatusTablet state changed. Reason: New leader 795d9d72a3ee4b98a8e2c986ce1d2168. Latest consensus state: current_term: 1 leader_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } }
I20260619 16:54:05.733932 1109 tablet_replica.cc:442] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:05.734364 1101 tablet_replica.cc:439] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } }
I20260619 16:54:05.734457 1101 tablet_replica.cc:442] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:54:05.734529 1108 mvcc.cc:205] Tried to move back new op lower bound from 7298613435317592064 to 7298613434883010560. Current Snapshot: MvccSnapshot[applied={T|T < 7298613435317592064}]
I20260619 16:54:05.734560 1111 mvcc.cc:205] Tried to move back new op lower bound from 7298613435317592064 to 7298613434883010560. Current Snapshot: MvccSnapshot[applied={T|T < 7298613435317592064}]
I20260619 16:54:05.734675 1083 tablet_replica.cc:439] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } }
I20260619 16:54:05.734756 1083 tablet_replica.cc:442] T af941d11fef24f56b471df3402e8b310 P 4b7ea2754a62449996c2ea9cdf2023cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:05.735162 1101 tablet_replica.cc:439] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } }
I20260619 16:54:05.735239 1101 tablet_replica.cc:442] T af941d11fef24f56b471df3402e8b310 P 795d9d72a3ee4b98a8e2c986ce1d2168: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:54:05.735562 1107 mvcc.cc:205] Tried to move back new op lower bound from 7298613435317592064 to 7298613434883010560. Current Snapshot: MvccSnapshot[applied={T|T < 7298613435317592064}]
I20260619 16:54:05.735641 1109 tablet_replica.cc:439] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } } }
I20260619 16:54:05.735698 1109 tablet_replica.cc:442] T af941d11fef24f56b471df3402e8b310 P bb2f823b184a42188c93a89bb9133798: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:05.740407 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.737652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42530 (local address 127.30.148.131:34971)
0619 16:54:05.737693 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:05.737695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:05.737767 (+ 72us) server_negotiation.cc:407] Connection header received
0619 16:54:05.737835 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:05.737835 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:05.737849 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:05.737900 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:05.738029 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.738456 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.739544 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.739861 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.739923 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.740100 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:05.740182 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:05.740252 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:05.740314 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 16:54:05.740334 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:54:05.745121 1115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.742700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:34873 (local address 127.30.148.130:46643)
0619 16:54:05.742740 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:05.742742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:05.742820 (+ 78us) server_negotiation.cc:407] Connection header received
0619 16:54:05.742882 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:05.742882 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:05.742889 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:05.742933 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:05.743052 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.743454 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.744464 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.744655 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.744704 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.744921 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:05.744943 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:05.744980 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:05.745062 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 16:54:05.745076 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:54:05.745314 1125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.742614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:46643 (local address 127.30.148.131:34873)
0619 16:54:05.742761 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:05.742771 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:05.742828 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:05.742937 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:05.742938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:05.742940 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:54:05.743000 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.743001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.743524 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.743524 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.744386 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.744388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.744734 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.744735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.744807 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.744824 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:54:05.745005 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:54:05.745007 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:54:05.745059 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 16:54:05.745096 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":88,"threads_started":1}
I20260619 16:54:05.754307 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.751816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:60817 (local address 127.30.148.131:34971)
0619 16:54:05.751849 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:05.751851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:05.751858 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:54:05.751898 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:05.751899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:05.751908 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:05.751958 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:05.752065 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.752737 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.753581 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.753911 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.753993 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.754084 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:05.754111 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:05.754167 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:05.754199 (+ 32us) server_negotiation.cc:299] Negotiation successful
0619 16:54:05.754233 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:05.754354 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:05.751641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:34971 (local address 127.30.148.130:60817)
0619 16:54:05.751788 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:05.751797 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:05.751845 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:05.751963 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:05.751963 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:05.751965 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:54:05.752002 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.752003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.752794 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.752795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.753442 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:05.753445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:05.753971 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:05.753972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:05.754020 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:05.754032 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:54:05.754167 (+ 135us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:54:05.754168 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 16:54:05.754197 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 16:54:05.754253 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260619 16:54:05.761772 574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:05.761891 574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:05.762743 574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8: Sending DeleteTablet for 3 replicas of tablet 5c86c3ac7b81493584ca177076eb324d
I20260619 16:54:05.763052 991 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c86c3ac7b81493584ca177076eb324d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:05 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:42514
I20260619 16:54:05.763051 851 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c86c3ac7b81493584ca177076eb324d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:05 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:56038
I20260619 16:54:05.763108 711 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c86c3ac7b81493584ca177076eb324d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:05 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:32896
I20260619 16:54:05.763269 1130 tablet_replica.cc:330] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798: stopping tablet replica
I20260619 16:54:05.763330 1131 tablet_replica.cc:330] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168: stopping tablet replica
I20260619 16:54:05.763360 1130 raft_consensus.cc:2229] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:05.763448 1130 raft_consensus.cc:2258] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:05.763536 574 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:05.763679 574 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:05.763983 1130 ts_tablet_manager.cc:1905] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:05.764813 991 tablet_service.cc:1467] Processing CreateTablet for tablet 005a79b7f9c34eea9f66be65e5af9a62 (DEFAULT_TABLE table=test-table [id=76d492e1c2524207b783149ec6af91e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:05.764931 991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 005a79b7f9c34eea9f66be65e5af9a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:05.765033 1132 tablet_replica.cc:330] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd: stopping tablet replica
I20260619 16:54:05.765259 1132 raft_consensus.cc:2229] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:05.765333 1132 raft_consensus.cc:2258] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:05.765709 1132 ts_tablet_manager.cc:1905] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:05.766057 1130 ts_tablet_manager.cc:1918] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:05.766201 1130 log.cc:1196] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/5c86c3ac7b81493584ca177076eb324d
I20260619 16:54:05.766523 1130 ts_tablet_manager.cc:1939] T 5c86c3ac7b81493584ca177076eb324d P bb2f823b184a42188c93a89bb9133798: Deleting consensus metadata
I20260619 16:54:05.766669 851 tablet_service.cc:1467] Processing CreateTablet for tablet 005a79b7f9c34eea9f66be65e5af9a62 (DEFAULT_TABLE table=test-table [id=76d492e1c2524207b783149ec6af91e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:05.766795 851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 005a79b7f9c34eea9f66be65e5af9a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:05.767409 1132 ts_tablet_manager.cc:1918] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:05.767465 1132 log.cc:1196] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/wal/wals/5c86c3ac7b81493584ca177076eb324d
I20260619 16:54:05.767755 1132 ts_tablet_manager.cc:1939] T 5c86c3ac7b81493584ca177076eb324d P 4b7ea2754a62449996c2ea9cdf2023cd: Deleting consensus metadata
I20260619 16:54:05.768213 1134 tablet_bootstrap.cc:492] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168: Bootstrap starting.
I20260619 16:54:05.768515 711 tablet_service.cc:1467] Processing CreateTablet for tablet 005a79b7f9c34eea9f66be65e5af9a62 (DEFAULT_TABLE table=test-table [id=76d492e1c2524207b783149ec6af91e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:05.768543 1134 tablet_bootstrap.cc:654] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:05.768611 711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 005a79b7f9c34eea9f66be65e5af9a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:05.769008 1134 tablet_bootstrap.cc:492] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168: No bootstrap required, opened a new log
I20260619 16:54:05.769050 1134 ts_tablet_manager.cc:1397] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:05.769152 1134 raft_consensus.cc:348] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.769203 1134 raft_consensus.cc:374] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:05.769222 1134 raft_consensus.cc:729] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 795d9d72a3ee4b98a8e2c986ce1d2168, State: Initialized, Role: FOLLOWER
I20260619 16:54:05.769263 1134 consensus_queue.cc:260] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.769342 1134 ts_tablet_manager.cc:1428] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:05.770015 559 catalog_manager.cc:4841] TS 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177): tablet 5c86c3ac7b81493584ca177076eb324d (table test-table [id=f1f51233d3cb4f7fae3fd521f337c95f]) successfully deleted
I20260619 16:54:05.764055 1131 raft_consensus.cc:2229] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:05.770176 1131 raft_consensus.cc:2258] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:05.770506 1131 ts_tablet_manager.cc:1905] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:05.770517 558 catalog_manager.cc:4841] TS bb2f823b184a42188c93a89bb9133798 (127.30.148.131:34971): tablet 5c86c3ac7b81493584ca177076eb324d (table test-table [id=f1f51233d3cb4f7fae3fd521f337c95f]) successfully deleted
I20260619 16:54:05.771970 1131 ts_tablet_manager.cc:1918] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:05.772038 1131 log.cc:1196] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/5c86c3ac7b81493584ca177076eb324d
I20260619 16:54:05.772329 1131 ts_tablet_manager.cc:1939] T 5c86c3ac7b81493584ca177076eb324d P 795d9d72a3ee4b98a8e2c986ce1d2168: Deleting consensus metadata
I20260619 16:54:05.772699 1133 tablet_bootstrap.cc:492] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798: Bootstrap starting.
I20260619 16:54:05.772713 559 catalog_manager.cc:4841] TS 795d9d72a3ee4b98a8e2c986ce1d2168 (127.30.148.130:46643): tablet 5c86c3ac7b81493584ca177076eb324d (table test-table [id=f1f51233d3cb4f7fae3fd521f337c95f]) successfully deleted
I20260619 16:54:05.773089 1133 tablet_bootstrap.cc:654] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:05.773445 1136 tablet_bootstrap.cc:492] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd: Bootstrap starting.
I20260619 16:54:05.773830 1133 tablet_bootstrap.cc:492] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798: No bootstrap required, opened a new log
I20260619 16:54:05.773881 1136 tablet_bootstrap.cc:654] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:05.773880 1133 ts_tablet_manager.cc:1397] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:05.774024 1133 raft_consensus.cc:348] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.774093 1133 raft_consensus.cc:374] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:05.774120 1133 raft_consensus.cc:729] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2f823b184a42188c93a89bb9133798, State: Initialized, Role: FOLLOWER
I20260619 16:54:05.774181 1133 consensus_queue.cc:260] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.774288 1133 ts_tablet_manager.cc:1428] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:05.777536 1136 tablet_bootstrap.cc:492] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd: No bootstrap required, opened a new log
I20260619 16:54:05.777599 1136 ts_tablet_manager.cc:1397] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 16:54:05.777751 1136 raft_consensus.cc:348] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.777830 1136 raft_consensus.cc:374] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:05.777853 1136 raft_consensus.cc:729] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b7ea2754a62449996c2ea9cdf2023cd, State: Initialized, Role: FOLLOWER
I20260619 16:54:05.777931 1136 consensus_queue.cc:260] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.778023 1136 ts_tablet_manager.cc:1428] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:05.937752 1106 raft_consensus.cc:482] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:05.937840 1106 raft_consensus.cc:504] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.937964 1106 leader_election.cc:288] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177), 795d9d72a3ee4b98a8e2c986ce1d2168 (127.30.148.130:46643)
I20260619 16:54:05.938215 731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005a79b7f9c34eea9f66be65e5af9a62" candidate_uuid: "bb2f823b184a42188c93a89bb9133798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" is_pre_election: true
I20260619 16:54:05.938251 871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005a79b7f9c34eea9f66be65e5af9a62" candidate_uuid: "bb2f823b184a42188c93a89bb9133798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" is_pre_election: true
I20260619 16:54:05.938326 731 raft_consensus.cc:2454] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb2f823b184a42188c93a89bb9133798 in term 0.
I20260619 16:54:05.938364 871 raft_consensus.cc:2454] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb2f823b184a42188c93a89bb9133798 in term 0.
I20260619 16:54:05.938483 943 leader_election.cc:302] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b7ea2754a62449996c2ea9cdf2023cd, bb2f823b184a42188c93a89bb9133798; no voters:
I20260619 16:54:05.938580 1106 raft_consensus.cc:2781] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:05.938621 1106 raft_consensus.cc:482] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:05.938647 1106 raft_consensus.cc:3037] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:05.939352 1106 raft_consensus.cc:504] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.939497 1106 leader_election.cc:288] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [CANDIDATE]: Term 1 election: Requested vote from peers 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177), 795d9d72a3ee4b98a8e2c986ce1d2168 (127.30.148.130:46643)
I20260619 16:54:05.939713 731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005a79b7f9c34eea9f66be65e5af9a62" candidate_uuid: "bb2f823b184a42188c93a89bb9133798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7ea2754a62449996c2ea9cdf2023cd"
I20260619 16:54:05.939706 871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005a79b7f9c34eea9f66be65e5af9a62" candidate_uuid: "bb2f823b184a42188c93a89bb9133798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168"
I20260619 16:54:05.939800 871 raft_consensus.cc:3037] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:05.939801 731 raft_consensus.cc:3037] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:05.940448 731 raft_consensus.cc:2454] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb2f823b184a42188c93a89bb9133798 in term 1.
I20260619 16:54:05.940546 871 raft_consensus.cc:2454] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb2f823b184a42188c93a89bb9133798 in term 1.
I20260619 16:54:05.940645 943 leader_election.cc:302] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b7ea2754a62449996c2ea9cdf2023cd, bb2f823b184a42188c93a89bb9133798; no voters:
I20260619 16:54:05.940747 1106 raft_consensus.cc:2781] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:05.940816 1106 raft_consensus.cc:686] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 1 LEADER]: Becoming Leader. State: Replica: bb2f823b184a42188c93a89bb9133798, State: Running, Role: LEADER
I20260619 16:54:05.940872 1106 consensus_queue.cc:237] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:05.941501 574 catalog_manager.cc:5495] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb2f823b184a42188c93a89bb9133798 (127.30.148.131). New cstate: current_term: 1 leader_uuid: "bb2f823b184a42188c93a89bb9133798" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } health_report { overall_health: HEALTHY } } }
I20260619 16:54:05.955304 871 raft_consensus.cc:1261] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Refusing update from remote peer bb2f823b184a42188c93a89bb9133798: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:05.955317 731 raft_consensus.cc:1261] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Refusing update from remote peer bb2f823b184a42188c93a89bb9133798: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:05.955499 1106 consensus_queue.cc:1035] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:05.955586 1109 consensus_queue.cc:1035] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 16:54:05.960192 1107 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 16:54:07.481817 573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:07.481912 573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:07.482715 573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8: Sending DeleteTablet for 3 replicas of tablet 005a79b7f9c34eea9f66be65e5af9a62
I20260619 16:54:07.482967 711 tablet_service.cc:1514] Processing DeleteTablet for tablet 005a79b7f9c34eea9f66be65e5af9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:07 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:32896
I20260619 16:54:07.482988 991 tablet_service.cc:1514] Processing DeleteTablet for tablet 005a79b7f9c34eea9f66be65e5af9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:07 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:42514
I20260619 16:54:07.483024 851 tablet_service.cc:1514] Processing DeleteTablet for tablet 005a79b7f9c34eea9f66be65e5af9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:07 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:56038
I20260619 16:54:07.483191 1165 tablet_replica.cc:330] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd: stopping tablet replica
I20260619 16:54:07.483253 1165 raft_consensus.cc:2229] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:07.483290 1165 raft_consensus.cc:2258] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:07.483330 1167 tablet_replica.cc:330] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168: stopping tablet replica
I20260619 16:54:07.483479 1167 raft_consensus.cc:2229] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:07.483527 1167 raft_consensus.cc:2258] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:07.483521 1165 ts_tablet_manager.cc:1905] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:07.483762 1167 ts_tablet_manager.cc:1905] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:07.485128 1165 ts_tablet_manager.cc:1918] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 16:54:07.485198 1165 log.cc:1196] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/wal/wals/005a79b7f9c34eea9f66be65e5af9a62
I20260619 16:54:07.485422 1167 ts_tablet_manager.cc:1918] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 16:54:07.485481 1165 ts_tablet_manager.cc:1939] T 005a79b7f9c34eea9f66be65e5af9a62 P 4b7ea2754a62449996c2ea9cdf2023cd: Deleting consensus metadata
I20260619 16:54:07.485489 1167 log.cc:1196] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/005a79b7f9c34eea9f66be65e5af9a62
I20260619 16:54:07.485482 1166 tablet_replica.cc:330] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798: stopping tablet replica
I20260619 16:54:07.485541 1166 raft_consensus.cc:2229] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:07.485610 1166 raft_consensus.cc:2258] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:07.485702 1167 ts_tablet_manager.cc:1939] T 005a79b7f9c34eea9f66be65e5af9a62 P 795d9d72a3ee4b98a8e2c986ce1d2168: Deleting consensus metadata
I20260619 16:54:07.485865 1166 ts_tablet_manager.cc:1905] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:07.485868 559 catalog_manager.cc:4841] TS 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177): tablet 005a79b7f9c34eea9f66be65e5af9a62 (table test-table [id=76d492e1c2524207b783149ec6af91e8]) successfully deleted
I20260619 16:54:07.486089 559 catalog_manager.cc:4841] TS 795d9d72a3ee4b98a8e2c986ce1d2168 (127.30.148.130:46643): tablet 005a79b7f9c34eea9f66be65e5af9a62 (table test-table [id=76d492e1c2524207b783149ec6af91e8]) successfully deleted
I20260619 16:54:07.487103 1166 ts_tablet_manager.cc:1918] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 16:54:07.487152 1166 log.cc:1196] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/005a79b7f9c34eea9f66be65e5af9a62
I20260619 16:54:07.487329 1166 ts_tablet_manager.cc:1939] T 005a79b7f9c34eea9f66be65e5af9a62 P bb2f823b184a42188c93a89bb9133798: Deleting consensus metadata
I20260619 16:54:07.487618 558 catalog_manager.cc:4841] TS bb2f823b184a42188c93a89bb9133798 (127.30.148.131:34971): tablet 005a79b7f9c34eea9f66be65e5af9a62 (table test-table [id=76d492e1c2524207b783149ec6af91e8]) successfully deleted
Jun 19 16:54:07 dist-test-slave-xb9n krb5kdc[529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888047, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 16:54:07 dist-test-slave-xb9n krb5kdc[529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888047, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:07.511165 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:07.501284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44084 (local address 127.30.148.190:41625)
0619 16:54:07.501435 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:07.501437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:07.501452 (+ 15us) server_negotiation.cc:407] Connection header received
0619 16:54:07.501513 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:07.501513 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:07.501524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:07.501589 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:07.502090 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:07.502578 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:07.503251 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:07.503446 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:07.503495 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:07.508951 (+ 5456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:07.508952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:07.508952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:07.508958 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:07.510286 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:07.510573 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:07.510573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:07.510574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:07.510623 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:07.510766 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:07.510767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:07.510767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:07.510858 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:07.510930 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:07.511010 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 16:54:07.511070 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":114,"threads_started":1}
I20260619 16:54:07.512562 573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:07.512712 573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:07.514169 851 tablet_service.cc:1467] Processing CreateTablet for tablet e62e938b3721460b83224145e932820f (DEFAULT_TABLE table=test-table [id=429c57865531461f8424100053fe8b32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:07.514171 991 tablet_service.cc:1467] Processing CreateTablet for tablet e62e938b3721460b83224145e932820f (DEFAULT_TABLE table=test-table [id=429c57865531461f8424100053fe8b32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:07.514178 711 tablet_service.cc:1467] Processing CreateTablet for tablet e62e938b3721460b83224145e932820f (DEFAULT_TABLE table=test-table [id=429c57865531461f8424100053fe8b32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:07.514298 991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e62e938b3721460b83224145e932820f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:07.514298 851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e62e938b3721460b83224145e932820f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:07.514340 711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e62e938b3721460b83224145e932820f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:07.515805 1177 tablet_bootstrap.cc:492] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798: Bootstrap starting.
I20260619 16:54:07.515805 1178 tablet_bootstrap.cc:492] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168: Bootstrap starting.
I20260619 16:54:07.515967 1176 tablet_bootstrap.cc:492] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd: Bootstrap starting.
I20260619 16:54:07.516135 1178 tablet_bootstrap.cc:654] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:07.516155 1177 tablet_bootstrap.cc:654] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:07.516373 1176 tablet_bootstrap.cc:654] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:07.516812 1177 tablet_bootstrap.cc:492] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798: No bootstrap required, opened a new log
I20260619 16:54:07.516916 1177 ts_tablet_manager.cc:1397] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:07.517022 1178 tablet_bootstrap.cc:492] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168: No bootstrap required, opened a new log
I20260619 16:54:07.517051 1177 raft_consensus.cc:348] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.517120 1178 ts_tablet_manager.cc:1397] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:07.517133 1177 raft_consensus.cc:374] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:07.517156 1177 raft_consensus.cc:729] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2f823b184a42188c93a89bb9133798, State: Initialized, Role: FOLLOWER
I20260619 16:54:07.517303 1178 raft_consensus.cc:348] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.517417 1178 raft_consensus.cc:374] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:07.517453 1176 tablet_bootstrap.cc:492] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd: No bootstrap required, opened a new log
I20260619 16:54:07.517488 1178 raft_consensus.cc:729] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 795d9d72a3ee4b98a8e2c986ce1d2168, State: Initialized, Role: FOLLOWER
I20260619 16:54:07.517508 1177 consensus_queue.cc:260] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.517545 1178 consensus_queue.cc:260] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.517570 1176 ts_tablet_manager.cc:1397] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:07.517689 1178 ts_tablet_manager.cc:1428] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:07.517730 1176 raft_consensus.cc:348] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.517817 1176 raft_consensus.cc:374] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:07.517867 1176 raft_consensus.cc:729] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b7ea2754a62449996c2ea9cdf2023cd, State: Initialized, Role: FOLLOWER
I20260619 16:54:07.517864 1177 ts_tablet_manager.cc:1428] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 16:54:07.517935 1176 consensus_queue.cc:260] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.518126 1176 ts_tablet_manager.cc:1428] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 16:54:07.530776 1151 raft_consensus.cc:482] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:07.530833 1151 raft_consensus.cc:504] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.530974 1151 leader_election.cc:288] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177), bb2f823b184a42188c93a89bb9133798 (127.30.148.131:34971)
I20260619 16:54:07.531121 731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e62e938b3721460b83224145e932820f" candidate_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" is_pre_election: true
I20260619 16:54:07.531111 1011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e62e938b3721460b83224145e932820f" candidate_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2f823b184a42188c93a89bb9133798" is_pre_election: true
I20260619 16:54:07.531191 1011 raft_consensus.cc:2454] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 795d9d72a3ee4b98a8e2c986ce1d2168 in term 0.
I20260619 16:54:07.531191 731 raft_consensus.cc:2454] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 795d9d72a3ee4b98a8e2c986ce1d2168 in term 0.
I20260619 16:54:07.531322 802 leader_election.cc:302] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 795d9d72a3ee4b98a8e2c986ce1d2168, bb2f823b184a42188c93a89bb9133798; no voters:
I20260619 16:54:07.531447 1151 raft_consensus.cc:2781] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:07.531493 1151 raft_consensus.cc:482] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:07.531508 1151 raft_consensus.cc:3037] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:07.532097 1151 raft_consensus.cc:504] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.532269 1151 leader_election.cc:288] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [CANDIDATE]: Term 1 election: Requested vote from peers 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177), bb2f823b184a42188c93a89bb9133798 (127.30.148.131:34971)
I20260619 16:54:07.532389 731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e62e938b3721460b83224145e932820f" candidate_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7ea2754a62449996c2ea9cdf2023cd"
I20260619 16:54:07.532409 1011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e62e938b3721460b83224145e932820f" candidate_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2f823b184a42188c93a89bb9133798"
I20260619 16:54:07.532472 1011 raft_consensus.cc:3037] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:07.532481 731 raft_consensus.cc:3037] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:07.533027 731 raft_consensus.cc:2454] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 795d9d72a3ee4b98a8e2c986ce1d2168 in term 1.
I20260619 16:54:07.533082 1011 raft_consensus.cc:2454] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 795d9d72a3ee4b98a8e2c986ce1d2168 in term 1.
I20260619 16:54:07.533243 803 leader_election.cc:302] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b7ea2754a62449996c2ea9cdf2023cd, 795d9d72a3ee4b98a8e2c986ce1d2168; no voters:
I20260619 16:54:07.533344 1151 raft_consensus.cc:2781] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:07.533427 1151 raft_consensus.cc:686] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 LEADER]: Becoming Leader. State: Replica: 795d9d72a3ee4b98a8e2c986ce1d2168, State: Running, Role: LEADER
I20260619 16:54:07.533496 1151 consensus_queue.cc:237] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.534057 574 catalog_manager.cc:5495] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 795d9d72a3ee4b98a8e2c986ce1d2168 (127.30.148.130). New cstate: current_term: 1 leader_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } health_report { overall_health: UNKNOWN } } }
I20260619 16:54:07.540392 1185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:07.538167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56056 (local address 127.30.148.130:46643)
0619 16:54:07.538305 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:07.538307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:07.538313 (+ 6us) server_negotiation.cc:407] Connection header received
0619 16:54:07.538331 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:07.538332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:07.538342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:07.538397 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:07.538483 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:07.538962 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:07.539811 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:07.540026 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:07.540079 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:07.540187 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:07.540215 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:07.540261 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:07.540321 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 16:54:07.540339 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260619 16:54:07.544250 1185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:07.542317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:48883 (local address 127.30.148.130:46643)
0619 16:54:07.542372 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:07.542374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:07.542480 (+ 106us) server_negotiation.cc:407] Connection header received
0619 16:54:07.542505 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:07.542505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:07.542514 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:07.542558 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:07.542661 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:07.543084 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:07.543738 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:07.543902 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:07.543949 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:07.544061 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:07.544081 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:07.544117 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:07.544163 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 16:54:07.544184 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":35}
I20260619 16:54:07.544325 1188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:07.542273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:46643 (local address 127.30.148.130:48883)
0619 16:54:07.542416 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:07.542425 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:07.542474 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:07.542578 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:07.542579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:07.542581 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:54:07.542612 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:07.542613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:07.543140 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:07.543140 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:07.543667 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:07.543668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:07.543961 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:07.543961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:07.543996 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:07.544000 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:54:07.544128 (+ 128us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:54:07.544129 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 16:54:07.544162 (+ 33us) client_negotiation.cc:240] Negotiation successful
0619 16:54:07.544202 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260619 16:54:07.546121 1011 raft_consensus.cc:1261] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Refusing update from remote peer 795d9d72a3ee4b98a8e2c986ce1d2168: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:07.546196 731 raft_consensus.cc:1261] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Refusing update from remote peer 795d9d72a3ee4b98a8e2c986ce1d2168: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:07.546288 1146 consensus_queue.cc:1035] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:07.546368 1151 consensus_queue.cc:1035] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:07.555922 574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:07.556013 574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:07.556602 574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8: Sending DeleteTablet for 3 replicas of tablet e62e938b3721460b83224145e932820f
I20260619 16:54:07.556835 991 tablet_service.cc:1514] Processing DeleteTablet for tablet e62e938b3721460b83224145e932820f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:07 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:42514
I20260619 16:54:07.556916 1166 tablet_replica.cc:330] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798: stopping tablet replica
I20260619 16:54:07.556887 711 tablet_service.cc:1514] Processing DeleteTablet for tablet e62e938b3721460b83224145e932820f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:07 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:32896
I20260619 16:54:07.556960 1166 raft_consensus.cc:2229] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:07.556975 1165 tablet_replica.cc:330] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd: stopping tablet replica
I20260619 16:54:07.556990 1166 raft_consensus.cc:2258] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:07.557030 1165 raft_consensus.cc:2229] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:07.557087 1165 raft_consensus.cc:2258] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:07.557196 574 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:07.557335 574 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:07.557408 851 tablet_service.cc:1514] Processing DeleteTablet for tablet e62e938b3721460b83224145e932820f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:07 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:56038
I20260619 16:54:07.557480 1165 ts_tablet_manager.cc:1905] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:07.557509 1167 tablet_replica.cc:330] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168: stopping tablet replica
I20260619 16:54:07.557554 1167 raft_consensus.cc:2229] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:07.557631 1167 raft_consensus.cc:2258] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:07.557889 1167 ts_tablet_manager.cc:1905] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:07.558914 991 tablet_service.cc:1467] Processing CreateTablet for tablet 3a1da0af35044808a377a3bd1e2b5e52 (DEFAULT_TABLE table=test-table [id=4fe03b2b3ad74b9ab4db44a07b5f8e3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:07.559017 991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a1da0af35044808a377a3bd1e2b5e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:07.559029 1165 ts_tablet_manager.cc:1918] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:07.559077 1165 log.cc:1196] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-0/wal/wals/e62e938b3721460b83224145e932820f
I20260619 16:54:07.559312 1165 ts_tablet_manager.cc:1939] T e62e938b3721460b83224145e932820f P 4b7ea2754a62449996c2ea9cdf2023cd: Deleting consensus metadata
I20260619 16:54:07.559477 1167 ts_tablet_manager.cc:1918] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:07.559522 1167 log.cc:1196] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/e62e938b3721460b83224145e932820f
I20260619 16:54:07.559697 1167 ts_tablet_manager.cc:1939] T e62e938b3721460b83224145e932820f P 795d9d72a3ee4b98a8e2c986ce1d2168: Deleting consensus metadata
I20260619 16:54:07.559994 559 catalog_manager.cc:4841] TS 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177): tablet e62e938b3721460b83224145e932820f (table test-table [id=429c57865531461f8424100053fe8b32]) successfully deleted
I20260619 16:54:07.560053 559 catalog_manager.cc:4841] TS 795d9d72a3ee4b98a8e2c986ce1d2168 (127.30.148.130:46643): tablet e62e938b3721460b83224145e932820f (table test-table [id=429c57865531461f8424100053fe8b32]) successfully deleted
I20260619 16:54:07.560119 711 tablet_service.cc:1467] Processing CreateTablet for tablet 3a1da0af35044808a377a3bd1e2b5e52 (DEFAULT_TABLE table=test-table [id=4fe03b2b3ad74b9ab4db44a07b5f8e3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:07.560195 1177 tablet_bootstrap.cc:492] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798: Bootstrap starting.
I20260619 16:54:07.560215 711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a1da0af35044808a377a3bd1e2b5e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:07.560498 1177 tablet_bootstrap.cc:654] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:07.560782 1166 ts_tablet_manager.cc:1905] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:07.561503 851 tablet_service.cc:1467] Processing CreateTablet for tablet 3a1da0af35044808a377a3bd1e2b5e52 (DEFAULT_TABLE table=test-table [id=4fe03b2b3ad74b9ab4db44a07b5f8e3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:07.561621 851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a1da0af35044808a377a3bd1e2b5e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:07.562141 1166 ts_tablet_manager.cc:1918] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:07.562186 1166 log.cc:1196] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/e62e938b3721460b83224145e932820f
I20260619 16:54:07.562384 1166 ts_tablet_manager.cc:1939] T e62e938b3721460b83224145e932820f P bb2f823b184a42188c93a89bb9133798: Deleting consensus metadata
I20260619 16:54:07.562608 1176 tablet_bootstrap.cc:492] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd: Bootstrap starting.
I20260619 16:54:07.562678 558 catalog_manager.cc:4841] TS bb2f823b184a42188c93a89bb9133798 (127.30.148.131:34971): tablet e62e938b3721460b83224145e932820f (table test-table [id=429c57865531461f8424100053fe8b32]) successfully deleted
I20260619 16:54:07.562840 1178 tablet_bootstrap.cc:492] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168: Bootstrap starting.
I20260619 16:54:07.562983 1176 tablet_bootstrap.cc:654] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:07.563094 1178 tablet_bootstrap.cc:654] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:07.563441 1177 tablet_bootstrap.cc:492] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798: No bootstrap required, opened a new log
I20260619 16:54:07.563491 1177 ts_tablet_manager.cc:1397] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 16:54:07.563588 1176 tablet_bootstrap.cc:492] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd: No bootstrap required, opened a new log
I20260619 16:54:07.563628 1176 ts_tablet_manager.cc:1397] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:07.563604 1177 raft_consensus.cc:348] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.563668 1177 raft_consensus.cc:374] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:07.563688 1177 raft_consensus.cc:729] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2f823b184a42188c93a89bb9133798, State: Initialized, Role: FOLLOWER
I20260619 16:54:07.563702 1178 tablet_bootstrap.cc:492] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168: No bootstrap required, opened a new log
I20260619 16:54:07.563740 1178 ts_tablet_manager.cc:1397] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 16:54:07.563728 1177 consensus_queue.cc:260] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.563750 1176 raft_consensus.cc:348] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.563807 1176 raft_consensus.cc:374] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:07.563812 1177 ts_tablet_manager.cc:1428] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:07.563825 1176 raft_consensus.cc:729] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b7ea2754a62449996c2ea9cdf2023cd, State: Initialized, Role: FOLLOWER
I20260619 16:54:07.563823 1178 raft_consensus.cc:348] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.563884 1178 raft_consensus.cc:374] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:07.563884 1176 consensus_queue.cc:260] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.563908 1178 raft_consensus.cc:729] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 795d9d72a3ee4b98a8e2c986ce1d2168, State: Initialized, Role: FOLLOWER
I20260619 16:54:07.563961 1176 ts_tablet_manager.cc:1428] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:07.563946 1178 consensus_queue.cc:260] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.564044 1178 ts_tablet_manager.cc:1428] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:07.924736 1101 raft_consensus.cc:482] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:07.924832 1101 raft_consensus.cc:504] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.924983 1101 leader_election.cc:288] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177), bb2f823b184a42188c93a89bb9133798 (127.30.148.131:34971)
I20260619 16:54:07.925191 1011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a1da0af35044808a377a3bd1e2b5e52" candidate_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2f823b184a42188c93a89bb9133798" is_pre_election: true
I20260619 16:54:07.925191 731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a1da0af35044808a377a3bd1e2b5e52" candidate_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" is_pre_election: true
I20260619 16:54:07.925298 731 raft_consensus.cc:2454] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 795d9d72a3ee4b98a8e2c986ce1d2168 in term 0.
I20260619 16:54:07.925298 1011 raft_consensus.cc:2454] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 795d9d72a3ee4b98a8e2c986ce1d2168 in term 0.
I20260619 16:54:07.925503 802 leader_election.cc:302] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 795d9d72a3ee4b98a8e2c986ce1d2168, bb2f823b184a42188c93a89bb9133798; no voters:
I20260619 16:54:07.925644 1101 raft_consensus.cc:2781] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:07.925712 1101 raft_consensus.cc:482] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:07.925755 1101 raft_consensus.cc:3037] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:07.926458 1101 raft_consensus.cc:504] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.926600 1101 leader_election.cc:288] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [CANDIDATE]: Term 1 election: Requested vote from peers 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177), bb2f823b184a42188c93a89bb9133798 (127.30.148.131:34971)
I20260619 16:54:07.926728 731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a1da0af35044808a377a3bd1e2b5e52" candidate_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7ea2754a62449996c2ea9cdf2023cd"
I20260619 16:54:07.926800 731 raft_consensus.cc:3037] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:07.926771 1011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a1da0af35044808a377a3bd1e2b5e52" candidate_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2f823b184a42188c93a89bb9133798"
I20260619 16:54:07.926874 1011 raft_consensus.cc:3037] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:07.927397 731 raft_consensus.cc:2454] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 795d9d72a3ee4b98a8e2c986ce1d2168 in term 1.
I20260619 16:54:07.927484 1011 raft_consensus.cc:2454] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 795d9d72a3ee4b98a8e2c986ce1d2168 in term 1.
I20260619 16:54:07.927569 803 leader_election.cc:302] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b7ea2754a62449996c2ea9cdf2023cd, 795d9d72a3ee4b98a8e2c986ce1d2168; no voters:
I20260619 16:54:07.927656 1101 raft_consensus.cc:2781] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:07.927706 1101 raft_consensus.cc:686] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 LEADER]: Becoming Leader. State: Replica: 795d9d72a3ee4b98a8e2c986ce1d2168, State: Running, Role: LEADER
I20260619 16:54:07.927758 1101 consensus_queue.cc:237] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } }
I20260619 16:54:07.928354 574 catalog_manager.cc:5495] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 reported cstate change: term changed from 0 to 1, leader changed from <none> to 795d9d72a3ee4b98a8e2c986ce1d2168 (127.30.148.130). New cstate: current_term: 1 leader_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "795d9d72a3ee4b98a8e2c986ce1d2168" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 46643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 } health_report { overall_health: UNKNOWN } } }
I20260619 16:54:08.002436 731 raft_consensus.cc:1261] T 3a1da0af35044808a377a3bd1e2b5e52 P 4b7ea2754a62449996c2ea9cdf2023cd [term 1 FOLLOWER]: Refusing update from remote peer 795d9d72a3ee4b98a8e2c986ce1d2168: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:08.002440 1011 raft_consensus.cc:1261] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Refusing update from remote peer 795d9d72a3ee4b98a8e2c986ce1d2168: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:08.002728 1101 consensus_queue.cc:1035] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b7ea2754a62449996c2ea9cdf2023cd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 33177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:08.002851 1151 consensus_queue.cc:1035] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb2f823b184a42188c93a89bb9133798" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 16:54:08.007007 1144 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 16:54:09.525868 572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:09.526005 572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:09.526818 572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63c6fa5d0f574a169f036c3f41f2d9d8: Sending DeleteTablet for 3 replicas of tablet 3a1da0af35044808a377a3bd1e2b5e52
I20260619 16:54:09.527087 991 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a1da0af35044808a377a3bd1e2b5e52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:09 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:42514
I20260619 16:54:09.527105 711 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a1da0af35044808a377a3bd1e2b5e52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:09 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:32896
I20260619 16:54:09.527117 850 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a1da0af35044808a377a3bd1e2b5e52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:09 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:56038
I20260619 16:54:09.527300 1218 tablet_replica.cc:330] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168: stopping tablet replica
I20260619 16:54:09.527410 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 646
I20260619 16:54:09.527478 1218 raft_consensus.cc:2229] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:09.528293 1218 raft_consensus.cc:2258] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:09.528407 1217 tablet_replica.cc:330] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798: stopping tablet replica
I20260619 16:54:09.528455 1217 raft_consensus.cc:2229] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:09.528494 1217 raft_consensus.cc:2258] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:09.528707 1218 ts_tablet_manager.cc:1905] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:09.528748 1217 ts_tablet_manager.cc:1905] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:09.530009 1218 ts_tablet_manager.cc:1918] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 16:54:09.530047 1218 log.cc:1196] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/3a1da0af35044808a377a3bd1e2b5e52
I20260619 16:54:09.530221 1218 ts_tablet_manager.cc:1939] T 3a1da0af35044808a377a3bd1e2b5e52 P 795d9d72a3ee4b98a8e2c986ce1d2168: Deleting consensus metadata
I20260619 16:54:09.530755 559 catalog_manager.cc:4841] TS 795d9d72a3ee4b98a8e2c986ce1d2168 (127.30.148.130:46643): tablet 3a1da0af35044808a377a3bd1e2b5e52 (table test-table [id=4fe03b2b3ad74b9ab4db44a07b5f8e3f]) successfully deleted
I20260619 16:54:09.530802 1217 ts_tablet_manager.cc:1918] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 16:54:09.530864 1217 log.cc:1196] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/3a1da0af35044808a377a3bd1e2b5e52
I20260619 16:54:09.531065 1217 ts_tablet_manager.cc:1939] T 3a1da0af35044808a377a3bd1e2b5e52 P bb2f823b184a42188c93a89bb9133798: Deleting consensus metadata
I20260619 16:54:09.531528 558 catalog_manager.cc:4841] TS bb2f823b184a42188c93a89bb9133798 (127.30.148.131:34971): tablet 3a1da0af35044808a377a3bd1e2b5e52 (table test-table [id=4fe03b2b3ad74b9ab4db44a07b5f8e3f]) successfully deleted
W20260619 16:54:09.533548 559 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:33177 (error 108)
I20260619 16:54:09.533813 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 786
I20260619 16:54:09.535131 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:09.534381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:33177 (local address 127.0.0.1:32904)
0619 16:54:09.535060 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:09.535102 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:33177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":58,"threads_started":1}
W20260619 16:54:09.535244 559 catalog_manager.cc:4584] TS 4b7ea2754a62449996c2ea9cdf2023cd (127.30.148.129:33177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3a1da0af35044808a377a3bd1e2b5e52: Network error: Client connection negotiation failed: client connection to 127.30.148.129:33177: connect: Connection refused (error 111)
I20260619 16:54:09.539556 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 926
I20260619 16:54:09.544606 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 542
2026-06-19T16:54:09Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6532 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:09 dist-test-slave-xb9n krb5kdc[1224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:09 dist-test-slave-xb9n krb5kdc[1224](info): set up 2 sockets
Jun 19 16:54:09 dist-test-slave-xb9n krb5kdc[1224](info): commencing operation
krb5kdc: starting...
W20260619 16:54:10.577103 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:10 dist-test-slave-xb9n krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:54:10.687420 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:37813
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:42481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:37813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:10.768994 1237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:10.769212 1237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:10.769248 1237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:10.770457 1237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:10.770525 1237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:10.770550 1237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:10.770624 1237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:10.770650 1237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:10.772235 1237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:42481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:37813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:37813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:10.772455 1237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:10.772706 1237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:10.777448 1243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:10.777503 1237 server_base.cc:1034] running on GCE node
W20260619 16:54:10.777441 1245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:10.777441 1242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:10.777895 1237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:10.778173 1237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:10.779351 1237 hybrid_clock.cc:648] HybridClock initialized: now 1781888050779342 us; error 58 us; skew 500 ppm
Jun 19 16:54:10 dist-test-slave-xb9n krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888050, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:10.781750 1237 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:54:10.782076 1237 webserver.cc:460] Webserver started at http://127.30.148.190:39715/ using document root <none> and password file <none>
I20260619 16:54:10.782236 1237 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:10.782272 1237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:10.782351 1237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:10.783221 1237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "3f5720a5d3d045be92aa9700daa9d9c6"
format_stamp: "Formatted at 2026-06-19 16:54:10 on dist-test-slave-xb9n"
server_key: "64e6fc8dc6f1e66006d983f9abb17728"
server_key_iv: "7aae1c2ffac04e0fdf099cae186fe2ff"
server_key_version: "encryptionkey@0"
I20260619 16:54:10.783485 1237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "3f5720a5d3d045be92aa9700daa9d9c6"
format_stamp: "Formatted at 2026-06-19 16:54:10 on dist-test-slave-xb9n"
server_key: "64e6fc8dc6f1e66006d983f9abb17728"
server_key_iv: "7aae1c2ffac04e0fdf099cae186fe2ff"
server_key_version: "encryptionkey@0"
I20260619 16:54:10.784636 1237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:10.785267 1252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:10.785470 1237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:10.785578 1237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "3f5720a5d3d045be92aa9700daa9d9c6"
format_stamp: "Formatted at 2026-06-19 16:54:10 on dist-test-slave-xb9n"
server_key: "64e6fc8dc6f1e66006d983f9abb17728"
server_key_iv: "7aae1c2ffac04e0fdf099cae186fe2ff"
server_key_version: "encryptionkey@0"
I20260619 16:54:10.785655 1237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:10.801229 1237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:10.803709 1237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:10.803889 1237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:10.808069 1237 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:37813
I20260619 16:54:10.808099 1304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:37813 every 8 connection(s)
I20260619 16:54:10.808447 1237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:10.809265 1305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:10.811434 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 1237
I20260619 16:54:10.811533 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:10.811542 1305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6: Bootstrap starting.
I20260619 16:54:10.811702 31314 external_mini_cluster.cc:1428] Setting key 4eccd6a7ecdbcc4a2cf3a9d3819b5d02
I20260619 16:54:10.812228 1305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:10.812531 1305 log.cc:824] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:10.813301 1305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6: No bootstrap required, opened a new log
I20260619 16:54:10.815341 1305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5720a5d3d045be92aa9700daa9d9c6" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 37813 } }
I20260619 16:54:10.815513 1305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:10.815542 1305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f5720a5d3d045be92aa9700daa9d9c6, State: Initialized, Role: FOLLOWER
I20260619 16:54:10.815660 1305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5720a5d3d045be92aa9700daa9d9c6" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 37813 } }
I20260619 16:54:10.815735 1305 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:10.815773 1305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:10.815850 1305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:10.816442 1305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5720a5d3d045be92aa9700daa9d9c6" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 37813 } }
I20260619 16:54:10.816560 1305 leader_election.cc:302] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f5720a5d3d045be92aa9700daa9d9c6; no voters:
I20260619 16:54:10.816730 1305 leader_election.cc:288] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:10.816797 1310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:10.816946 1310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [term 1 LEADER]: Becoming Leader. State: Replica: 3f5720a5d3d045be92aa9700daa9d9c6, State: Running, Role: LEADER
I20260619 16:54:10.817081 1305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:10.817118 1310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5720a5d3d045be92aa9700daa9d9c6" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 37813 } }
I20260619 16:54:10.817435 1310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f5720a5d3d045be92aa9700daa9d9c6. Latest consensus state: current_term: 1 leader_uuid: "3f5720a5d3d045be92aa9700daa9d9c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5720a5d3d045be92aa9700daa9d9c6" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 37813 } } }
I20260619 16:54:10.817443 1311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f5720a5d3d045be92aa9700daa9d9c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f5720a5d3d045be92aa9700daa9d9c6" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 37813 } } }
I20260619 16:54:10.817610 1310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:10.817725 1311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:10.818482 1317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 16:54:10 dist-test-slave-xb9n krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:10.818976 1317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:10.820407 1317 catalog_manager.cc:1347] Generated new cluster ID: 6b61e93a5a4b41bd9b298b30f8629227
I20260619 16:54:10.820500 1317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:10.823019 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:10.812154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.30.148.190:37813)
0619 16:54:10.812305 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:10.812314 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:10.812344 (+ 30us) server_negotiation.cc:407] Connection header received
0619 16:54:10.812920 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:10.812928 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:10.812988 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:10.813110 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:10.813865 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:10.814678 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:10.815772 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:10.816256 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:10.816332 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:10.819248 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:10.819252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:10.819252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:10.819262 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:10.821552 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:10.821884 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:10.821887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:10.821888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:10.821982 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:10.822193 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:10.822194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:10.822194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:10.822418 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:10.822578 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:10.822689 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 16:54:10.822821 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260619 16:54:10.831178 1317 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:10.831641 1317 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:10.834306 1317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f5720a5d3d045be92aa9700daa9d9c6: Generated new TSK 0
I20260619 16:54:10.834474 1317 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:54:10.859200 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:37813
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:42481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:10.953532 1331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:10.953720 1331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:10.953750 1331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:10.954875 1331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:10.954913 1331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:10.954937 1331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:10.956262 1331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:42481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:37813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:10.956502 1331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:10.956759 1331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:10.962327 1339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:10.962335 1337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:10.962555 1331 server_base.cc:1034] running on GCE node
W20260619 16:54:10.962349 1336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:10.962828 1331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:10.963092 1331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:10.964265 1331 hybrid_clock.cc:648] HybridClock initialized: now 1781888050964252 us; error 31 us; skew 500 ppm
Jun 19 16:54:10 dist-test-slave-xb9n krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888050, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:10.966982 1331 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:54:10.967429 1331 webserver.cc:460] Webserver started at http://127.30.148.129:35171/ using document root <none> and password file <none>
I20260619 16:54:10.967674 1331 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:10.967751 1331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:10.967898 1331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:10.968900 1331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "3bf2b0037f35496ea10b9248d9fbbc64"
format_stamp: "Formatted at 2026-06-19 16:54:10 on dist-test-slave-xb9n"
server_key: "7cb47fcf06c5a33590fb90bb38f1ede5"
server_key_iv: "a6118bfd638eef69c37f286df4fd9468"
server_key_version: "encryptionkey@0"
I20260619 16:54:10.969266 1331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "3bf2b0037f35496ea10b9248d9fbbc64"
format_stamp: "Formatted at 2026-06-19 16:54:10 on dist-test-slave-xb9n"
server_key: "7cb47fcf06c5a33590fb90bb38f1ede5"
server_key_iv: "a6118bfd638eef69c37f286df4fd9468"
server_key_version: "encryptionkey@0"
I20260619 16:54:10.970556 1331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:10.971220 1346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:10.971379 1331 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:10.971474 1331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "3bf2b0037f35496ea10b9248d9fbbc64"
format_stamp: "Formatted at 2026-06-19 16:54:10 on dist-test-slave-xb9n"
server_key: "7cb47fcf06c5a33590fb90bb38f1ede5"
server_key_iv: "a6118bfd638eef69c37f286df4fd9468"
server_key_version: "encryptionkey@0"
I20260619 16:54:10.971553 1331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:10.990011 1331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:10.992142 1331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:10.992295 1331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:10.992496 1331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:10.992813 1331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:10.992868 1331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:10.992906 1331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:10.992934 1331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:10.998705 1331 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:34043
I20260619 16:54:10.998747 1459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:34043 every 8 connection(s)
I20260619 16:54:10.999043 1331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:11.004065 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 1331
I20260619 16:54:11.004137 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:11.004323 31314 external_mini_cluster.cc:1428] Setting key 569e55e52cef891fbad1ba9112dbc7cf
Jun 19 16:54:11 dist-test-slave-xb9n krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888050, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:11.007762 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:10.999745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:35301 (local address 127.30.148.190:37813)
0619 16:54:10.999807 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:10.999810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:11.000051 (+ 241us) server_negotiation.cc:407] Connection header received
0619 16:54:11.000449 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:11.000450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:11.000460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:11.000523 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:11.001330 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:11.001861 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:11.002757 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:11.002938 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:11.002988 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:11.004997 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:11.004998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:11.004998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:11.005006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:11.006380 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:11.006786 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:11.006787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:11.006788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:11.006841 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:11.007101 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:11.007102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:11.007103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:11.007215 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:11.007288 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:11.007585 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 16:54:11.007674 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:54:11.007970 1462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:10.999739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:37813 (local address 127.30.148.129:35301)
0619 16:54:10.999922 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:10.999952 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:11.000374 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:11.000572 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:11.000575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:11.001063 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:11.001242 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:11.001246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:11.001929 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:11.001930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:11.002676 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:11.002678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:11.003004 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:11.003005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:11.003098 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:11.003505 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:11.003512 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:11.004901 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:11.006474 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:11.006480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:11.006482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:11.006697 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:11.006913 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:11.006913 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:11.006914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:11.007018 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:11.007280 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:11.007286 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:11.007419 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 16:54:11.007554 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 16:54:11.007704 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260619 16:54:11.008436 1460 heartbeater.cc:347] Connected to a master server at 127.30.148.190:37813
I20260619 16:54:11.008512 1460 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:11.008677 1460 heartbeater.cc:511] Master 127.30.148.190:37813 requested a full tablet report, sending...
I20260619 16:54:11.009163 1269 ts_manager.cc:194] Registered new tserver with Master: 3bf2b0037f35496ea10b9248d9fbbc64 (127.30.148.129:34043)
I20260619 16:54:11.009791 1269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:35301
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:54:11.031191 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:37813
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:42481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:11.109037 1465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:11.109251 1465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:11.109287 1465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:11.110451 1465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:11.110524 1465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:11.110570 1465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:11.111984 1465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:42481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:37813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:11.112248 1465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:11.112457 1465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:11.117188 1471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:11.117223 1465 server_base.cc:1034] running on GCE node
W20260619 16:54:11.117208 1470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:11.117201 1473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:11.117671 1465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:11.117929 1465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:11.119110 1465 hybrid_clock.cc:648] HybridClock initialized: now 1781888051119103 us; error 39 us; skew 500 ppm
Jun 19 16:54:11 dist-test-slave-xb9n krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888051, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:11.121413 1465 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:54:11.121879 1465 webserver.cc:460] Webserver started at http://127.30.148.130:33419/ using document root <none> and password file <none>
I20260619 16:54:11.122095 1465 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:11.122138 1465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:11.122282 1465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:11.123265 1465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "3112b1a373ce49b98cee4d7e1d6d2cc2"
format_stamp: "Formatted at 2026-06-19 16:54:11 on dist-test-slave-xb9n"
server_key: "01b96764bf17e8b8b037b3cdcf7d587b"
server_key_iv: "85c3348ab2b241a1f42eb231f622d06e"
server_key_version: "encryptionkey@0"
I20260619 16:54:11.123615 1465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "3112b1a373ce49b98cee4d7e1d6d2cc2"
format_stamp: "Formatted at 2026-06-19 16:54:11 on dist-test-slave-xb9n"
server_key: "01b96764bf17e8b8b037b3cdcf7d587b"
server_key_iv: "85c3348ab2b241a1f42eb231f622d06e"
server_key_version: "encryptionkey@0"
I20260619 16:54:11.124850 1465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 16:54:11.125568 1480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:11.125741 1465 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:11.125818 1465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "3112b1a373ce49b98cee4d7e1d6d2cc2"
format_stamp: "Formatted at 2026-06-19 16:54:11 on dist-test-slave-xb9n"
server_key: "01b96764bf17e8b8b037b3cdcf7d587b"
server_key_iv: "85c3348ab2b241a1f42eb231f622d06e"
server_key_version: "encryptionkey@0"
I20260619 16:54:11.125869 1465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:11.139166 1465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:11.141286 1465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:11.141469 1465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:11.141630 1465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:11.141882 1465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:11.141911 1465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:11.141932 1465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:11.141943 1465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:11.147279 1465 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:40959
I20260619 16:54:11.147341 1593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:40959 every 8 connection(s)
I20260619 16:54:11.147645 1465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:54:11 dist-test-slave-xb9n krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888051, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:11.155531 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 1465
I20260619 16:54:11.155637 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:11.155835 31314 external_mini_cluster.cc:1428] Setting key 2b934d4e953dc2929a1d99e7e5577251
I20260619 16:54:11.156136 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:11.148338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:52961 (local address 127.30.148.190:37813)
0619 16:54:11.148419 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:11.148421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:11.148568 (+ 147us) server_negotiation.cc:407] Connection header received
0619 16:54:11.148990 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:11.148991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:11.149001 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:11.149053 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:11.149938 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:11.150430 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:11.151376 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:11.151573 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:11.151625 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:11.153642 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:11.153644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:11.153644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:11.153650 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:11.154800 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:11.155126 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:11.155127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:11.155127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:11.155172 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:11.155428 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:11.155428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:11.155428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:11.155532 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:11.155579 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:11.155921 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 16:54:11.156027 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 16:54:11.156240 1596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:11.148282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:37813 (local address 127.30.148.130:52961)
0619 16:54:11.148458 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:11.148490 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:11.148910 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:11.149102 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:11.149105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:11.149682 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:11.149856 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:11.149860 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:11.150507 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:11.150508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:11.151293 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:11.151295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:11.151642 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:11.151642 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:11.151742 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:11.152210 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:11.152224 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:11.153565 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:11.154867 (+ 1302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:11.154872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:11.154874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:11.155066 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:11.155233 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:11.155233 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:11.155234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:11.155327 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:11.155597 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:11.155603 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:11.155759 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 16:54:11.155874 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 16:54:11.156030 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260619 16:54:11.156787 1594 heartbeater.cc:347] Connected to a master server at 127.30.148.190:37813
I20260619 16:54:11.156862 1594 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:11.157055 1594 heartbeater.cc:511] Master 127.30.148.190:37813 requested a full tablet report, sending...
I20260619 16:54:11.157423 1269 ts_manager.cc:194] Registered new tserver with Master: 3112b1a373ce49b98cee4d7e1d6d2cc2 (127.30.148.130:40959)
I20260619 16:54:11.157799 1269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:52961
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:54:11.179211 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:37813
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:42481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:11.256824 1599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:11.257006 1599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:11.257035 1599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:11.258168 1599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:11.258206 1599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:11.258267 1599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:11.259620 1599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:42481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:37813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:11.259850 1599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:11.260082 1599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:11.264760 1605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:11.264780 1607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:11.264853 1604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:11.264814 1599 server_base.cc:1034] running on GCE node
I20260619 16:54:11.265240 1599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:11.265518 1599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:11.266664 1599 hybrid_clock.cc:648] HybridClock initialized: now 1781888051266654 us; error 27 us; skew 500 ppm
Jun 19 16:54:11 dist-test-slave-xb9n krb5kdc[1224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888051, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:11.269124 1599 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:54:11.269616 1599 webserver.cc:460] Webserver started at http://127.30.148.131:45525/ using document root <none> and password file <none>
I20260619 16:54:11.269861 1599 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:11.269927 1599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:11.270059 1599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:11.271035 1599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "a67719a63bfa450b8b958f21e61b317e"
format_stamp: "Formatted at 2026-06-19 16:54:11 on dist-test-slave-xb9n"
server_key: "26167233c97e285e4632f3e812d6b20c"
server_key_iv: "6bc2333113ed6408519ad9e095f07b8e"
server_key_version: "encryptionkey@0"
I20260619 16:54:11.271428 1599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "a67719a63bfa450b8b958f21e61b317e"
format_stamp: "Formatted at 2026-06-19 16:54:11 on dist-test-slave-xb9n"
server_key: "26167233c97e285e4632f3e812d6b20c"
server_key_iv: "6bc2333113ed6408519ad9e095f07b8e"
server_key_version: "encryptionkey@0"
I20260619 16:54:11.272734 1599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:11.273427 1614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:11.273602 1599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:11.273691 1599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "a67719a63bfa450b8b958f21e61b317e"
format_stamp: "Formatted at 2026-06-19 16:54:11 on dist-test-slave-xb9n"
server_key: "26167233c97e285e4632f3e812d6b20c"
server_key_iv: "6bc2333113ed6408519ad9e095f07b8e"
server_key_version: "encryptionkey@0"
I20260619 16:54:11.273744 1599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:11.300839 1599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:11.303144 1599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:11.303309 1599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:11.303514 1599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:11.303803 1599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:11.303857 1599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:11.303902 1599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:11.303926 1599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:11.309552 1599 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:41159
I20260619 16:54:11.309605 1727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:41159 every 8 connection(s)
I20260619 16:54:11.309885 1599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:54:11.313573 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 1599
I20260619 16:54:11.313704 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:11.313922 31314 external_mini_cluster.cc:1428] Setting key 0c3c5819e35402746c18d9c238fc9826
Jun 19 16:54:11 dist-test-slave-xb9n krb5kdc[1224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888051, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:11.319005 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:11.310542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:48851 (local address 127.30.148.190:37813)
0619 16:54:11.310602 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:11.310604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:11.310799 (+ 195us) server_negotiation.cc:407] Connection header received
0619 16:54:11.311223 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:11.311224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:11.311232 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:11.311287 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:11.312127 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:11.312593 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:11.313624 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:11.313912 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:11.313968 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:11.316081 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:11.316082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:11.316082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:11.316087 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:11.317623 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:11.318087 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:11.318088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:11.318088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:11.318134 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:11.318375 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:11.318375 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:11.318375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:11.318516 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:11.318573 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:11.318872 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 16:54:11.318945 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 16:54:11.319109 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:11.310516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:37813 (local address 127.30.148.131:48851)
0619 16:54:11.310674 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:11.310712 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:11.311147 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:11.311338 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:11.311341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:11.311878 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:11.312054 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:11.312058 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:11.312666 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:11.312666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:11.313536 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:11.313539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:11.314001 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:11.314002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:11.314163 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:11.314685 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:11.314693 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:11.316003 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:11.317688 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:11.317694 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:11.317695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:11.318030 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:11.318189 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:11.318190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:11.318190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:11.318319 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:11.318567 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:11.318571 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:11.318722 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 16:54:11.318831 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 16:54:11.318943 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260619 16:54:11.319590 1728 heartbeater.cc:347] Connected to a master server at 127.30.148.190:37813
I20260619 16:54:11.319662 1728 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:11.319782 1728 heartbeater.cc:511] Master 127.30.148.190:37813 requested a full tablet report, sending...
I20260619 16:54:11.320072 1269 ts_manager.cc:194] Registered new tserver with Master: a67719a63bfa450b8b958f21e61b317e (127.30.148.131:41159)
I20260619 16:54:11.320456 1269 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:48851
I20260619 16:54:11.325609 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 16:54:11.336102 1308 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:11.335196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33628 (local address 127.30.148.190:37813)
0619 16:54:11.335252 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:11.335254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:11.335387 (+ 133us) server_negotiation.cc:407] Connection header received
0619 16:54:11.335475 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:11.335476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:11.335486 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:11.335558 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:11.336039 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33628: BlockingRecv error: recv got EOF from 127.0.0.1:33628 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 16:54:11.336903 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 1331
I20260619 16:54:11.341416 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 1465
I20260619 16:54:11.345815 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 1599
I20260619 16:54:11.349923 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 1237
2026-06-19T16:54:11Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1803 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:11 dist-test-slave-xb9n krb5kdc[1740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:11 dist-test-slave-xb9n krb5kdc[1740](info): set up 2 sockets
Jun 19 16:54:11 dist-test-slave-xb9n krb5kdc[1740](info): commencing operation
krb5kdc: starting...
W20260619 16:54:12.377058 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:12 dist-test-slave-xb9n krb5kdc[1740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:54:12.487833 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:44049
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:44049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:12.566316 1753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:12.566556 1753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:12.566593 1753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:12.567929 1753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:12.567991 1753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:12.568014 1753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:12.568037 1753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:12.568065 1753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:12.569584 1753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:44049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:44049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:12.569852 1753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:12.570072 1753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:12.574894 1758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:12.574916 1759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:12.574898 1761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:12.574945 1753 server_base.cc:1034] running on GCE node
I20260619 16:54:12.575323 1753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:12.575546 1753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:12.576680 1753 hybrid_clock.cc:648] HybridClock initialized: now 1781888052576670 us; error 27 us; skew 500 ppm
Jun 19 16:54:12 dist-test-slave-xb9n krb5kdc[1740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888052, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:12.583031 1753 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:54:12.583448 1753 webserver.cc:460] Webserver started at http://127.30.148.190:34923/ using document root <none> and password file <none>
I20260619 16:54:12.583670 1753 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:12.583715 1753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:12.583853 1753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:12.584911 1753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "32976bdb05614814aeab1dc83ab3a8f7"
format_stamp: "Formatted at 2026-06-19 16:54:12 on dist-test-slave-xb9n"
server_key: "d16e84a9d67bd66b3955c27ccc682c32"
server_key_iv: "760e8d99c447c9c97b91d02f594bbb2d"
server_key_version: "encryptionkey@0"
I20260619 16:54:12.585283 1753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "32976bdb05614814aeab1dc83ab3a8f7"
format_stamp: "Formatted at 2026-06-19 16:54:12 on dist-test-slave-xb9n"
server_key: "d16e84a9d67bd66b3955c27ccc682c32"
server_key_iv: "760e8d99c447c9c97b91d02f594bbb2d"
server_key_version: "encryptionkey@0"
I20260619 16:54:12.586592 1753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:12.587286 1768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:12.587471 1753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:12.587579 1753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "32976bdb05614814aeab1dc83ab3a8f7"
format_stamp: "Formatted at 2026-06-19 16:54:12 on dist-test-slave-xb9n"
server_key: "d16e84a9d67bd66b3955c27ccc682c32"
server_key_iv: "760e8d99c447c9c97b91d02f594bbb2d"
server_key_version: "encryptionkey@0"
I20260619 16:54:12.587647 1753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:12.607689 1753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:12.610036 1753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:12.610203 1753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:12.614068 1753 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:44049
I20260619 16:54:12.614120 1820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:44049 every 8 connection(s)
I20260619 16:54:12.614589 1753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:12.615258 1821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:12.617501 1821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7: Bootstrap starting.
I20260619 16:54:12.618047 1821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:12.618290 1821 log.cc:824] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:12.618907 1821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7: No bootstrap required, opened a new log
I20260619 16:54:12.620388 1821 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32976bdb05614814aeab1dc83ab3a8f7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44049 } }
I20260619 16:54:12.620522 1821 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:12.620544 1821 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32976bdb05614814aeab1dc83ab3a8f7, State: Initialized, Role: FOLLOWER
I20260619 16:54:12.620678 1821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32976bdb05614814aeab1dc83ab3a8f7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44049 } }
I20260619 16:54:12.620762 1821 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:12.620786 1821 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:12.620813 1821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:12.621726 1821 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32976bdb05614814aeab1dc83ab3a8f7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44049 } }
I20260619 16:54:12.621814 1821 leader_election.cc:302] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32976bdb05614814aeab1dc83ab3a8f7; no voters:
I20260619 16:54:12.621961 1821 leader_election.cc:288] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:12.622004 1824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:12.622169 1824 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [term 1 LEADER]: Becoming Leader. State: Replica: 32976bdb05614814aeab1dc83ab3a8f7, State: Running, Role: LEADER
I20260619 16:54:12.622141 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 1753
I20260619 16:54:12.622263 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:12.622236 1821 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:12.622422 31314 external_mini_cluster.cc:1428] Setting key fb44ae83fc51fc41137fe856e6420618
I20260619 16:54:12.623054 1824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32976bdb05614814aeab1dc83ab3a8f7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44049 } }
I20260619 16:54:12.623768 1826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32976bdb05614814aeab1dc83ab3a8f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32976bdb05614814aeab1dc83ab3a8f7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44049 } } }
I20260619 16:54:12.623692 1825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32976bdb05614814aeab1dc83ab3a8f7. Latest consensus state: current_term: 1 leader_uuid: "32976bdb05614814aeab1dc83ab3a8f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32976bdb05614814aeab1dc83ab3a8f7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44049 } } }
I20260619 16:54:12.623849 1826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:12.623854 1825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7 [sys.catalog]: This master's current role is: LEADER
W20260619 16:54:12.624181 1840 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:54:12.624286 1840 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:54:12.624402 1841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:12.624595 1841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:12.626894 1841 catalog_manager.cc:1347] Generated new cluster ID: e6b63ba9515a4fe0a88ae2099621f37b
I20260619 16:54:12.626947 1841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:54:12 dist-test-slave-xb9n krb5kdc[1740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:12.634567 1833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:12.622882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34584 (local address 127.30.148.190:44049)
0619 16:54:12.623082 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:12.623090 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:12.623300 (+ 210us) server_negotiation.cc:407] Connection header received
0619 16:54:12.623425 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:12.623433 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:12.624037 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:12.624251 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:12.625029 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:12.626409 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:12.627400 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:12.628063 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:12.628177 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:12.631018 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:12.631022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:12.631022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:12.631033 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:12.632931 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:12.633344 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:12.633346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:12.633347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:12.633463 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:12.633680 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:12.633681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:12.633681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:12.633978 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:12.634111 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:12.634243 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 16:54:12.634363 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":97,"threads_started":1}
I20260619 16:54:12.635638 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:54:12.636572 1841 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:12.637305 1841 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:12.642185 1841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32976bdb05614814aeab1dc83ab3a8f7: Generated new TSK 0
I20260619 16:54:12.642475 1841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:54:12.654724 1833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:12.650841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34598 (local address 127.30.148.190:44049)
0619 16:54:12.650908 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:12.650911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:12.651019 (+ 108us) server_negotiation.cc:407] Connection header received
0619 16:54:12.651083 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:12.651084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:12.651104 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:12.651155 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:12.651630 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:12.652154 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:12.653244 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:12.653524 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:12.653577 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:12.654259 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:12.654260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:12.654260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:12.654439 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:12.654454 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:12.654460 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:12.654506 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:12.654592 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 16:54:12.654623 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 16:54:12.657191 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 1753
2026-06-19T16:54:12Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1305 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:12 dist-test-slave-xb9n krb5kdc[1854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:12 dist-test-slave-xb9n krb5kdc[1854](info): set up 2 sockets
Jun 19 16:54:12 dist-test-slave-xb9n krb5kdc[1854](info): commencing operation
krb5kdc: starting...
W20260619 16:54:13.685199 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:13 dist-test-slave-xb9n krb5kdc[1854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:54:13.796130 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:35823
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:33889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:35823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:13.874822 1867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:13.875036 1867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:13.875089 1867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:13.876251 1867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:13.876314 1867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:13.876338 1867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:13.876358 1867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:13.876386 1867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:13.877849 1867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:33889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:35823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:35823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:13.878123 1867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:13.878350 1867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:13.882978 1873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:13.883030 1875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:13.882962 1867 server_base.cc:1034] running on GCE node
W20260619 16:54:13.882966 1872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:13.883417 1867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:13.883648 1867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:13.884788 1867 hybrid_clock.cc:648] HybridClock initialized: now 1781888053884764 us; error 41 us; skew 500 ppm
Jun 19 16:54:13 dist-test-slave-xb9n krb5kdc[1854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888053, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:13.887154 1867 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:54:13.887532 1867 webserver.cc:460] Webserver started at http://127.30.148.190:33119/ using document root <none> and password file <none>
I20260619 16:54:13.887761 1867 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:13.887830 1867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:13.887979 1867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:13.888984 1867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "bf623dfdec064850bb3f187dddbeef3c"
format_stamp: "Formatted at 2026-06-19 16:54:13 on dist-test-slave-xb9n"
server_key: "5aaee409833efb17f96d32dafffafe44"
server_key_iv: "f5b1f862b058ca272957d61468bea8b2"
server_key_version: "encryptionkey@0"
I20260619 16:54:13.889405 1867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "bf623dfdec064850bb3f187dddbeef3c"
format_stamp: "Formatted at 2026-06-19 16:54:13 on dist-test-slave-xb9n"
server_key: "5aaee409833efb17f96d32dafffafe44"
server_key_iv: "f5b1f862b058ca272957d61468bea8b2"
server_key_version: "encryptionkey@0"
I20260619 16:54:13.890762 1867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:13.891448 1882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:13.891616 1867 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:13.891712 1867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "bf623dfdec064850bb3f187dddbeef3c"
format_stamp: "Formatted at 2026-06-19 16:54:13 on dist-test-slave-xb9n"
server_key: "5aaee409833efb17f96d32dafffafe44"
server_key_iv: "f5b1f862b058ca272957d61468bea8b2"
server_key_version: "encryptionkey@0"
I20260619 16:54:13.891803 1867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:13.902436 1867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:13.904126 1867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:13.904284 1867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:13.908185 1867 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:35823
I20260619 16:54:13.908234 1934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:35823 every 8 connection(s)
I20260619 16:54:13.908577 1867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:13.909576 1935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:13.910177 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 1867
I20260619 16:54:13.910276 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:13.910447 31314 external_mini_cluster.cc:1428] Setting key 7084ce23a914d13dd34718f0d5d0d46e
I20260619 16:54:13.912209 1935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c: Bootstrap starting.
I20260619 16:54:13.912757 1935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:13.913053 1935 log.cc:824] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:13.913744 1935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c: No bootstrap required, opened a new log
I20260619 16:54:13.915585 1935 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf623dfdec064850bb3f187dddbeef3c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 35823 } }
I20260619 16:54:13.915750 1935 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:13.915800 1935 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf623dfdec064850bb3f187dddbeef3c, State: Initialized, Role: FOLLOWER
I20260619 16:54:13.915927 1935 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf623dfdec064850bb3f187dddbeef3c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 35823 } }
I20260619 16:54:13.916041 1935 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:13.916074 1935 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:13.916116 1935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [term 0 FOLLOWER]: Advancing to term 1
Jun 19 16:54:13 dist-test-slave-xb9n krb5kdc[1854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:13.916786 1935 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf623dfdec064850bb3f187dddbeef3c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 35823 } }
I20260619 16:54:13.916891 1935 leader_election.cc:302] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf623dfdec064850bb3f187dddbeef3c; no voters:
I20260619 16:54:13.917047 1935 leader_election.cc:288] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:13.917106 1940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:13.917310 1935 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:13.917296 1940 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [term 1 LEADER]: Becoming Leader. State: Replica: bf623dfdec064850bb3f187dddbeef3c, State: Running, Role: LEADER
I20260619 16:54:13.917954 1940 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf623dfdec064850bb3f187dddbeef3c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 35823 } }
I20260619 16:54:13.918341 1942 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf623dfdec064850bb3f187dddbeef3c. Latest consensus state: current_term: 1 leader_uuid: "bf623dfdec064850bb3f187dddbeef3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf623dfdec064850bb3f187dddbeef3c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 35823 } } }
I20260619 16:54:13.918443 1942 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:13.918642 1953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:13.918919 1941 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf623dfdec064850bb3f187dddbeef3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf623dfdec064850bb3f187dddbeef3c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 35823 } } }
I20260619 16:54:13.918993 1941 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:13.919217 1953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:13.921571 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:13.910893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50034 (local address 127.30.148.190:35823)
0619 16:54:13.911092 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:13.911107 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:13.911131 (+ 24us) server_negotiation.cc:407] Connection header received
0619 16:54:13.911518 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:13.911525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:13.911566 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:13.911716 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:13.912351 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:13.913220 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:13.914068 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:13.914474 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:13.914592 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:13.917205 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:13.917209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:13.917209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:13.917221 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:13.919959 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:13.920404 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:13.920407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:13.920407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:13.920499 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:13.920716 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:13.920717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:13.920717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:13.920948 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:13.921088 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:13.921202 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 16:54:13.921330 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260619 16:54:13.921831 1953 catalog_manager.cc:1347] Generated new cluster ID: 296097cf3aa747d9a6e510715911f479
I20260619 16:54:13.921881 1953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:13.977860 1953 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:13.978588 1953 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:13.996146 1953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf623dfdec064850bb3f187dddbeef3c: Generated new TSK 0
I20260619 16:54:13.996342 1953 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:54:14.030220 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:35823
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:33889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:14.108729 1961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:14.108922 1961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:14.108943 1961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:14.110029 1961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:14.110065 1961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:14.110090 1961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:14.111418 1961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:33889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:35823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:14.111586 1961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:14.111760 1961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:14.116640 1966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:14.116657 1967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:14.116680 1969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:14.116961 1961 server_base.cc:1034] running on GCE node
I20260619 16:54:14.117161 1961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:14.117480 1961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:14.118616 1961 hybrid_clock.cc:648] HybridClock initialized: now 1781888054118592 us; error 39 us; skew 500 ppm
Jun 19 16:54:14 dist-test-slave-xb9n krb5kdc[1854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888054, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:14.121069 1961 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:54:14.121554 1961 webserver.cc:460] Webserver started at http://127.30.148.129:33503/ using document root <none> and password file <none>
I20260619 16:54:14.121776 1961 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:14.121843 1961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:14.122008 1961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:14.123041 1961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "ea3a302213ad4272af01f223480130c7"
format_stamp: "Formatted at 2026-06-19 16:54:14 on dist-test-slave-xb9n"
server_key: "5072e1ea62a552cbea7cf0b8a800d700"
server_key_iv: "f8e5aacc13b865f4cd83d36ad04d9b02"
server_key_version: "encryptionkey@0"
I20260619 16:54:14.123409 1961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "ea3a302213ad4272af01f223480130c7"
format_stamp: "Formatted at 2026-06-19 16:54:14 on dist-test-slave-xb9n"
server_key: "5072e1ea62a552cbea7cf0b8a800d700"
server_key_iv: "f8e5aacc13b865f4cd83d36ad04d9b02"
server_key_version: "encryptionkey@0"
I20260619 16:54:14.124682 1961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:14.125339 1976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:14.125531 1961 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:14.125626 1961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "ea3a302213ad4272af01f223480130c7"
format_stamp: "Formatted at 2026-06-19 16:54:14 on dist-test-slave-xb9n"
server_key: "5072e1ea62a552cbea7cf0b8a800d700"
server_key_iv: "f8e5aacc13b865f4cd83d36ad04d9b02"
server_key_version: "encryptionkey@0"
I20260619 16:54:14.125706 1961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:14.138921 1961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:14.140595 1961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:14.140755 1961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:14.140961 1961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:14.141260 1961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:14.141315 1961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:14.141373 1961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:14.141405 1961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:14.147400 1961 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:39865
I20260619 16:54:14.147435 2089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:39865 every 8 connection(s)
I20260619 16:54:14.147768 1961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
Jun 19 16:54:14 dist-test-slave-xb9n krb5kdc[1854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888054, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:14.154574 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 1961
I20260619 16:54:14.154706 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:14.154918 31314 external_mini_cluster.cc:1428] Setting key 7a58cbc0488f78e1c056da92822afd2a
I20260619 16:54:14.156958 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:14.148475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:54289 (local address 127.30.148.190:35823)
0619 16:54:14.148541 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:14.148543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:14.148701 (+ 158us) server_negotiation.cc:407] Connection header received
0619 16:54:14.149158 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:14.149158 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:14.149167 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:14.149229 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:14.150151 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.150640 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.151602 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.151793 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.151843 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:14.153907 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:14.153908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:14.153908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:14.153914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:14.155558 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:14.156034 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:14.156035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:14.156035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:14.156086 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:14.156366 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:14.156367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:14.156367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:14.156463 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:14.156512 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:14.156823 (+ 311us) server_negotiation.cc:299] Negotiation successful
0619 16:54:14.156893 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 16:54:14.157244 2092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:14.148407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:35823 (local address 127.30.148.129:54289)
0619 16:54:14.148573 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:14.148612 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:14.149078 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:14.149272 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:14.149275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:14.149898 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:14.150073 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:14.150077 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.150743 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.150743 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:14.151529 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:14.151531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.151865 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.151865 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:14.151959 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:14.152469 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:14.152477 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:14.153831 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:14.155639 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:14.155646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:14.155648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:14.155962 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:14.156151 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:14.156152 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:14.156153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:14.156303 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:14.156526 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:14.156531 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:14.156689 (+ 158us) client_negotiation.cc:769] Sending connection context
0619 16:54:14.156805 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 16:54:14.156974 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260619 16:54:14.157824 2090 heartbeater.cc:347] Connected to a master server at 127.30.148.190:35823
I20260619 16:54:14.157933 2090 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:14.158123 2090 heartbeater.cc:511] Master 127.30.148.190:35823 requested a full tablet report, sending...
I20260619 16:54:14.158638 1899 ts_manager.cc:194] Registered new tserver with Master: ea3a302213ad4272af01f223480130c7 (127.30.148.129:39865)
I20260619 16:54:14.159546 1899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:54289
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:54:14.179314 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:35823
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:33889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:14.258533 2095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:14.258719 2095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:14.258744 2095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:14.259814 2095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:14.259848 2095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:14.259874 2095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:14.261191 2095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:33889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:35823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:14.261405 2095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:14.261595 2095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:14.266215 2101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:14.266261 2095 server_base.cc:1034] running on GCE node
W20260619 16:54:14.266191 2100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:14.266197 2103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:14.266628 2095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:14.266899 2095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:14.268088 2095 hybrid_clock.cc:648] HybridClock initialized: now 1781888054268082 us; error 33 us; skew 500 ppm
Jun 19 16:54:14 dist-test-slave-xb9n krb5kdc[1854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888054, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:14.270624 2095 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:54:14.271093 2095 webserver.cc:460] Webserver started at http://127.30.148.130:43005/ using document root <none> and password file <none>
I20260619 16:54:14.271313 2095 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:14.271356 2095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:14.271443 2095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:14.272393 2095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "bfe30af0f13f497987ab053e8bf85267"
format_stamp: "Formatted at 2026-06-19 16:54:14 on dist-test-slave-xb9n"
server_key: "ed878f0aee0e6cd876c01fc7681e3cf3"
server_key_iv: "b055264b35fdf24af5887a0fa02b1956"
server_key_version: "encryptionkey@0"
I20260619 16:54:14.272686 2095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "bfe30af0f13f497987ab053e8bf85267"
format_stamp: "Formatted at 2026-06-19 16:54:14 on dist-test-slave-xb9n"
server_key: "ed878f0aee0e6cd876c01fc7681e3cf3"
server_key_iv: "b055264b35fdf24af5887a0fa02b1956"
server_key_version: "encryptionkey@0"
I20260619 16:54:14.274042 2095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:14.274822 2110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:14.275061 2095 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:14.275166 2095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "bfe30af0f13f497987ab053e8bf85267"
format_stamp: "Formatted at 2026-06-19 16:54:14 on dist-test-slave-xb9n"
server_key: "ed878f0aee0e6cd876c01fc7681e3cf3"
server_key_iv: "b055264b35fdf24af5887a0fa02b1956"
server_key_version: "encryptionkey@0"
I20260619 16:54:14.275236 2095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:14.286943 2095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:14.288725 2095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:14.288872 2095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:14.289075 2095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:14.289397 2095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:14.289438 2095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:14.289490 2095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:14.289518 2095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:14.295289 2095 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:41389
I20260619 16:54:14.295397 2223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:41389 every 8 connection(s)
I20260619 16:54:14.295686 2095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:54:14 dist-test-slave-xb9n krb5kdc[1854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888054, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:14.304131 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 2095
I20260619 16:54:14.304248 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:14.304448 31314 external_mini_cluster.cc:1428] Setting key c7ada520c42446f25cea35ed423416d9
I20260619 16:54:14.304446 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:14.296385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:34957 (local address 127.30.148.190:35823)
0619 16:54:14.296453 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:14.296455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:14.296633 (+ 178us) server_negotiation.cc:407] Connection header received
0619 16:54:14.297110 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:14.297110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:14.297119 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:14.297174 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:14.298111 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.298600 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.299517 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.299695 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.299747 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:14.301814 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:14.301815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:14.301816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:14.301822 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:14.303196 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:14.303585 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:14.303586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:14.303586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:14.303634 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:14.303829 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:14.303830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:14.303830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:14.303922 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:14.304016 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:14.304268 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 16:54:14.304348 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 16:54:14.304653 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:14.296363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:35823 (local address 127.30.148.130:34957)
0619 16:54:14.296502 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:14.296553 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:14.297025 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:14.297222 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:14.297225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:14.297837 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:14.298022 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:14.298026 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.298676 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.298677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:14.299442 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:14.299444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.299782 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.299783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:14.299891 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:14.300334 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:14.300341 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:14.301730 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:14.303283 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:14.303288 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:14.303290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:14.303490 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:14.303685 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:14.303686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:14.303686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:14.303776 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:14.304003 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:14.304006 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:14.304114 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 16:54:14.304238 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 16:54:14.304394 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260619 16:54:14.305133 2224 heartbeater.cc:347] Connected to a master server at 127.30.148.190:35823
I20260619 16:54:14.305235 2224 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:14.305467 2224 heartbeater.cc:511] Master 127.30.148.190:35823 requested a full tablet report, sending...
I20260619 16:54:14.305841 1899 ts_manager.cc:194] Registered new tserver with Master: bfe30af0f13f497987ab053e8bf85267 (127.30.148.130:41389)
I20260619 16:54:14.306221 1899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:34957
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:54:14.328066 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:35823
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:33889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:14.406482 2229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:14.406706 2229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:14.406742 2229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:14.407855 2229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:14.407917 2229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:14.407961 2229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:14.409338 2229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:33889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:35823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:14.409672 2229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:14.409886 2229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:14.414724 2234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:14.414723 2237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:14.414891 2229 server_base.cc:1034] running on GCE node
W20260619 16:54:14.414745 2235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:14.415153 2229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:14.415424 2229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:14.416584 2229 hybrid_clock.cc:648] HybridClock initialized: now 1781888054416580 us; error 35 us; skew 500 ppm
Jun 19 16:54:14 dist-test-slave-xb9n krb5kdc[1854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888054, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:14.419296 2229 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:54:14.419704 2229 webserver.cc:460] Webserver started at http://127.30.148.131:38805/ using document root <none> and password file <none>
I20260619 16:54:14.419925 2229 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:14.419970 2229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:14.420113 2229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:14.421113 2229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "ceb2b03092a748bda00c631333dd13a1"
format_stamp: "Formatted at 2026-06-19 16:54:14 on dist-test-slave-xb9n"
server_key: "f2e62931aa8d9e8aac36256382dc2339"
server_key_iv: "fab921fc241248cbc8c4af18eb6e3e04"
server_key_version: "encryptionkey@0"
I20260619 16:54:14.421532 2229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "ceb2b03092a748bda00c631333dd13a1"
format_stamp: "Formatted at 2026-06-19 16:54:14 on dist-test-slave-xb9n"
server_key: "f2e62931aa8d9e8aac36256382dc2339"
server_key_iv: "fab921fc241248cbc8c4af18eb6e3e04"
server_key_version: "encryptionkey@0"
I20260619 16:54:14.422891 2229 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:14.423605 2244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:14.423765 2229 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:14.423862 2229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "ceb2b03092a748bda00c631333dd13a1"
format_stamp: "Formatted at 2026-06-19 16:54:14 on dist-test-slave-xb9n"
server_key: "f2e62931aa8d9e8aac36256382dc2339"
server_key_iv: "fab921fc241248cbc8c4af18eb6e3e04"
server_key_version: "encryptionkey@0"
I20260619 16:54:14.423934 2229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:14.444455 2229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:14.446352 2229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:14.446508 2229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:14.446724 2229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:14.447031 2229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:14.447085 2229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:14.447126 2229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:14.447154 2229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:14.452912 2229 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:44171
I20260619 16:54:14.452962 2357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:44171 every 8 connection(s)
I20260619 16:54:14.453254 2229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:54:14 dist-test-slave-xb9n krb5kdc[1854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888054, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:14.461889 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:14.453964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:47417 (local address 127.30.148.190:35823)
0619 16:54:14.454055 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:14.454058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:14.454204 (+ 146us) server_negotiation.cc:407] Connection header received
0619 16:54:14.454662 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:14.454663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:14.454673 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:14.454731 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:14.455590 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.456090 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.457012 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.457190 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.457241 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:14.459267 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:14.459269 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:14.459269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:14.459276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:14.460653 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:14.461006 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:14.461007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:14.461007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:14.461053 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:14.461281 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:14.461283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:14.461283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:14.461458 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:14.461521 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:14.461767 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 16:54:14.461827 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 16:54:14.462051 2360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:14.453917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:35823 (local address 127.30.148.131:47417)
0619 16:54:14.454074 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:14.454114 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:14.454570 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:14.454783 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:14.454786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:14.455334 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:14.455513 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:14.455518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.456165 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.456165 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:14.456933 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:14.456935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.457265 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.457266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:14.457434 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:14.457842 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:14.457849 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:14.459182 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:14.460728 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:14.460732 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:14.460734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:14.460928 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:14.461107 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:14.461108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:14.461108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:14.461192 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:14.461519 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:14.461523 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:14.461647 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 16:54:14.461741 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 16:54:14.461850 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260619 16:54:14.462478 2358 heartbeater.cc:347] Connected to a master server at 127.30.148.190:35823
I20260619 16:54:14.462565 2358 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:14.462692 2358 heartbeater.cc:511] Master 127.30.148.190:35823 requested a full tablet report, sending...
I20260619 16:54:14.462764 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 2229
I20260619 16:54:14.462848 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:14.463019 31314 external_mini_cluster.cc:1428] Setting key d8cc031b80a7b4a0861c0f49a8f60913
I20260619 16:54:14.463035 1899 ts_manager.cc:194] Registered new tserver with Master: ceb2b03092a748bda00c631333dd13a1 (127.30.148.131:44171)
I20260619 16:54:14.463511 1899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:47417
I20260619 16:54:14.463819 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 16:54:14 dist-test-slave-xb9n krb5kdc[1854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888054, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 16:54:14 dist-test-slave-xb9n krb5kdc[1854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888054, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:14.486207 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:14.479251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.30.148.190:35823)
0619 16:54:14.479319 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:14.479321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:14.479423 (+ 102us) server_negotiation.cc:407] Connection header received
0619 16:54:14.479479 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:14.479480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:14.479489 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:14.479526 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:14.480038 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.480502 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.481159 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:14.481330 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:14.481466 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:14.483873 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:14.483874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:14.483874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:14.483881 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:14.485228 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:14.485588 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:14.485589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:14.485589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:14.485634 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:14.485786 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:14.485786 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:14.485786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:14.485908 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:14.485977 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:14.486058 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 16:54:14.486112 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260619 16:54:14.486464 1898 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50038
I20260619 16:54:14.486867 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 1961
I20260619 16:54:14.492061 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 2095
I20260619 16:54:14.496225 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 2229
I20260619 16:54:14.500536 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 1867
2026-06-19T16:54:14Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1844 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:14 dist-test-slave-xb9n krb5kdc[2370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:14 dist-test-slave-xb9n krb5kdc[2370](info): set up 2 sockets
Jun 19 16:54:14 dist-test-slave-xb9n krb5kdc[2370](info): commencing operation
krb5kdc: starting...
W20260619 16:54:15.537240 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:15 dist-test-slave-xb9n krb5kdc[2370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:54:15.647503 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:44677
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:35525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:44677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:15.725314 2383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:15.725557 2383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:15.725596 2383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:15.727078 2383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:15.727140 2383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:15.727165 2383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:15.727196 2383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:15.727216 2383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:15.729537 2383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:44677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:44677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:15.729772 2383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:15.729995 2383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:15.734594 2391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:15.734649 2388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:15.734719 2383 server_base.cc:1034] running on GCE node
W20260619 16:54:15.734647 2389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:15.735062 2383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:15.735352 2383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:15.736490 2383 hybrid_clock.cc:648] HybridClock initialized: now 1781888055736475 us; error 26 us; skew 500 ppm
Jun 19 16:54:15 dist-test-slave-xb9n krb5kdc[2370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:15.738900 2383 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:54:15.739342 2383 webserver.cc:460] Webserver started at http://127.30.148.190:41883/ using document root <none> and password file <none>
I20260619 16:54:15.739567 2383 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:15.739610 2383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:15.739775 2383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:15.740773 2383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "b754aac772874515a94c7257d670ad28"
format_stamp: "Formatted at 2026-06-19 16:54:15 on dist-test-slave-xb9n"
server_key: "a404c0fe242da83005aaf45b2c1168d2"
server_key_iv: "ffcad0cf0bd1f2f8e0facb71ec8f73ff"
server_key_version: "encryptionkey@0"
I20260619 16:54:15.741117 2383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "b754aac772874515a94c7257d670ad28"
format_stamp: "Formatted at 2026-06-19 16:54:15 on dist-test-slave-xb9n"
server_key: "a404c0fe242da83005aaf45b2c1168d2"
server_key_iv: "ffcad0cf0bd1f2f8e0facb71ec8f73ff"
server_key_version: "encryptionkey@0"
I20260619 16:54:15.742440 2383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:54:15.743131 2398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:15.743310 2383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:15.743371 2383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "b754aac772874515a94c7257d670ad28"
format_stamp: "Formatted at 2026-06-19 16:54:15 on dist-test-slave-xb9n"
server_key: "a404c0fe242da83005aaf45b2c1168d2"
server_key_iv: "ffcad0cf0bd1f2f8e0facb71ec8f73ff"
server_key_version: "encryptionkey@0"
I20260619 16:54:15.743453 2383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:15.759778 2383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:15.761584 2383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:15.761734 2383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:15.765494 2383 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:44677
I20260619 16:54:15.765548 2450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:44677 every 8 connection(s)
I20260619 16:54:15.765996 2383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:15.766602 2451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:15.768718 2451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28: Bootstrap starting.
I20260619 16:54:15.769301 2451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:15.769614 2451 log.cc:824] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:15.770273 2451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28: No bootstrap required, opened a new log
I20260619 16:54:15.771831 2451 raft_consensus.cc:348] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b754aac772874515a94c7257d670ad28" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44677 } }
I20260619 16:54:15.772013 2451 raft_consensus.cc:374] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:15.772056 2451 raft_consensus.cc:729] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b754aac772874515a94c7257d670ad28, State: Initialized, Role: FOLLOWER
I20260619 16:54:15.772105 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 2383
I20260619 16:54:15.772200 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:15.772217 2451 consensus_queue.cc:260] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b754aac772874515a94c7257d670ad28" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44677 } }
I20260619 16:54:15.772295 2451 raft_consensus.cc:388] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:15.772338 2451 raft_consensus.cc:482] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:15.772385 2451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:15.772395 31314 external_mini_cluster.cc:1428] Setting key 8e2eead40e07821a2f80de71063b42f8
I20260619 16:54:15.773274 2451 raft_consensus.cc:504] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b754aac772874515a94c7257d670ad28" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44677 } }
I20260619 16:54:15.773406 2451 leader_election.cc:302] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b754aac772874515a94c7257d670ad28; no voters:
I20260619 16:54:15.773571 2451 leader_election.cc:288] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:15.773644 2456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:15.773815 2456 raft_consensus.cc:686] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [term 1 LEADER]: Becoming Leader. State: Replica: b754aac772874515a94c7257d670ad28, State: Running, Role: LEADER
I20260619 16:54:15.773844 2451 sys_catalog.cc:564] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:15.773906 2456 consensus_queue.cc:237] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b754aac772874515a94c7257d670ad28" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44677 } }
I20260619 16:54:15.774235 2456 sys_catalog.cc:455] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b754aac772874515a94c7257d670ad28. Latest consensus state: current_term: 1 leader_uuid: "b754aac772874515a94c7257d670ad28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b754aac772874515a94c7257d670ad28" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44677 } } }
I20260619 16:54:15.774243 2457 sys_catalog.cc:455] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b754aac772874515a94c7257d670ad28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b754aac772874515a94c7257d670ad28" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44677 } } }
I20260619 16:54:15.774405 2456 sys_catalog.cc:458] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:15.774413 2457 sys_catalog.cc:458] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:15.774634 2465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:15.775336 2465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:15.776909 2465 catalog_manager.cc:1347] Generated new cluster ID: 18b938bd818346b68aca3ebe61badd71
I20260619 16:54:15.776962 2465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:54:15 dist-test-slave-xb9n krb5kdc[2370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:15.784425 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:15.772931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46988 (local address 127.30.148.190:44677)
0619 16:54:15.773097 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:15.773105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:15.773132 (+ 27us) server_negotiation.cc:407] Connection header received
0619 16:54:15.773231 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:15.773235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:15.773273 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:15.773435 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:15.774267 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:15.775815 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:15.776849 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:15.777519 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:15.777632 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:15.780348 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:15.780352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:15.780352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:15.780363 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:15.782801 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:15.783212 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:15.783215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:15.783215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:15.783309 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:15.783520 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:15.783521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:15.783521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:15.783812 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:15.783942 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:15.784102 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 16:54:15.784225 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260619 16:54:15.792291 2465 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:15.792968 2465 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:15.800156 2465 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b754aac772874515a94c7257d670ad28: Generated new TSK 0
I20260619 16:54:15.800323 2465 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:54:15.827967 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:44677
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:35525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:15.906963 2477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:15.907203 2477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:15.907241 2477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:15.908396 2477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:15.908466 2477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:15.908512 2477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:15.909979 2477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:44677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:15.910251 2477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:15.910468 2477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:15.915354 2485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:15.915355 2482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:15.915380 2477 server_base.cc:1034] running on GCE node
W20260619 16:54:15.915380 2483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:15.915776 2477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:15.916029 2477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:15.917193 2477 hybrid_clock.cc:648] HybridClock initialized: now 1781888055917179 us; error 28 us; skew 500 ppm
Jun 19 16:54:15 dist-test-slave-xb9n krb5kdc[2370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:15.919636 2477 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:54:15.920014 2477 webserver.cc:460] Webserver started at http://127.30.148.129:38569/ using document root <none> and password file <none>
I20260619 16:54:15.920184 2477 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:15.920230 2477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:15.920315 2477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:15.921319 2477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "e915410d0f6f4288bb3fb5d4d4ab2dca"
format_stamp: "Formatted at 2026-06-19 16:54:15 on dist-test-slave-xb9n"
server_key: "d45a5b28d5ff59c8f4b421464d1ba222"
server_key_iv: "b7502a745e1c7c2c7efabee61270b2b9"
server_key_version: "encryptionkey@0"
I20260619 16:54:15.921736 2477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "e915410d0f6f4288bb3fb5d4d4ab2dca"
format_stamp: "Formatted at 2026-06-19 16:54:15 on dist-test-slave-xb9n"
server_key: "d45a5b28d5ff59c8f4b421464d1ba222"
server_key_iv: "b7502a745e1c7c2c7efabee61270b2b9"
server_key_version: "encryptionkey@0"
I20260619 16:54:15.923066 2477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 16:54:15.923804 2492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:15.923982 2477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:15.924084 2477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "e915410d0f6f4288bb3fb5d4d4ab2dca"
format_stamp: "Formatted at 2026-06-19 16:54:15 on dist-test-slave-xb9n"
server_key: "d45a5b28d5ff59c8f4b421464d1ba222"
server_key_iv: "b7502a745e1c7c2c7efabee61270b2b9"
server_key_version: "encryptionkey@0"
I20260619 16:54:15.924154 2477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:15.942088 2477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:15.944329 2477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:15.944485 2477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:15.944715 2477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:15.945022 2477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:15.945076 2477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:15.945115 2477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:15.945143 2477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:15.951123 2477 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:35021
I20260619 16:54:15.951167 2605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:35021 every 8 connection(s)
I20260619 16:54:15.951460 2477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:15.952664 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 2477
I20260619 16:54:15.952769 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:15.952972 31314 external_mini_cluster.cc:1428] Setting key fe707102ffd573e2de9e0b6c67318808
Jun 19 16:54:15 dist-test-slave-xb9n krb5kdc[2370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
I20260619 16:54:15.961925 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:15.952104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:47063 (local address 127.30.148.190:44677)
0619 16:54:15.952180 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:15.952183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:15.952369 (+ 186us) server_negotiation.cc:407] Connection header received
0619 16:54:15.953674 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:15.953675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:15.953687 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:15.953753 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:15.954598 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:15.955111 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:15.956108 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:15.956431 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:15.956504 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:15.958741 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:15.958742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:15.958743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:15.958751 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:15.960405 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:15.960828 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:15.960829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:15.960829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:15.960891 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:15.961130 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:15.961131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:15.961131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:15.961273 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:15.961350 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:15.961712 (+ 362us) server_negotiation.cc:299] Negotiation successful
0619 16:54:15.961796 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 16:54:15.961954 2608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:15.952084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:44677 (local address 127.30.148.129:47063)
0619 16:54:15.952256 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:15.952290 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:15.953552 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:15.953839 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:15.953843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:15.954220 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:15.954492 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:15.954508 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:15.955213 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:15.955214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:15.956016 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:15.956018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:15.956538 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:15.956539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:15.956653 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:15.957178 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:15.957193 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:15.958637 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:15.960523 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:15.960529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:15.960531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:15.960741 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:15.960963 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:15.960963 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:15.960963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:15.961057 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:15.961342 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:15.961347 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:15.961541 (+ 194us) client_negotiation.cc:769] Sending connection context
0619 16:54:15.961629 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 16:54:15.961804 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260619 16:54:15.962404 2606 heartbeater.cc:347] Connected to a master server at 127.30.148.190:44677
I20260619 16:54:15.962517 2606 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:15.962692 2606 heartbeater.cc:511] Master 127.30.148.190:44677 requested a full tablet report, sending...
I20260619 16:54:15.963143 2415 ts_manager.cc:194] Registered new tserver with Master: e915410d0f6f4288bb3fb5d4d4ab2dca (127.30.148.129:35021)
I20260619 16:54:15.963717 2415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:47063
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:54:15.977254 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:44677
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:35525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:16.055258 2611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:16.055449 2611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:16.055471 2611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:16.056608 2611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:16.056650 2611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:16.056703 2611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:16.058192 2611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:44677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:16.058441 2611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:16.058662 2611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:16.063529 2617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:16.063530 2616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:16.063637 2611 server_base.cc:1034] running on GCE node
W20260619 16:54:16.063530 2619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:16.063884 2611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:16.064143 2611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:16.065281 2611 hybrid_clock.cc:648] HybridClock initialized: now 1781888056065275 us; error 30 us; skew 500 ppm
Jun 19 16:54:16 dist-test-slave-xb9n krb5kdc[2370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:16.067840 2611 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:54:16.068212 2611 webserver.cc:460] Webserver started at http://127.30.148.130:45387/ using document root <none> and password file <none>
I20260619 16:54:16.068390 2611 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:16.068428 2611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:16.068517 2611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:16.069546 2611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "98b436bbe4474293b9469373863b2f8e"
format_stamp: "Formatted at 2026-06-19 16:54:16 on dist-test-slave-xb9n"
server_key: "58a64b18e73fbcf2e5234050bbef6682"
server_key_iv: "61fc56a4eafedd5831d35f98fe67e689"
server_key_version: "encryptionkey@0"
I20260619 16:54:16.069830 2611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "98b436bbe4474293b9469373863b2f8e"
format_stamp: "Formatted at 2026-06-19 16:54:16 on dist-test-slave-xb9n"
server_key: "58a64b18e73fbcf2e5234050bbef6682"
server_key_iv: "61fc56a4eafedd5831d35f98fe67e689"
server_key_version: "encryptionkey@0"
I20260619 16:54:16.071144 2611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:16.071810 2626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:16.071978 2611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:16.072057 2611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "98b436bbe4474293b9469373863b2f8e"
format_stamp: "Formatted at 2026-06-19 16:54:16 on dist-test-slave-xb9n"
server_key: "58a64b18e73fbcf2e5234050bbef6682"
server_key_iv: "61fc56a4eafedd5831d35f98fe67e689"
server_key_version: "encryptionkey@0"
I20260619 16:54:16.072108 2611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:16.082427 2611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:16.084161 2611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:16.084277 2611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:16.084463 2611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:16.084749 2611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:16.084781 2611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:16.084801 2611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:16.084836 2611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:16.090472 2611 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:45261
I20260619 16:54:16.090567 2739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:45261 every 8 connection(s)
I20260619 16:54:16.090850 2611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:54:16.091437 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 2611
I20260619 16:54:16.091540 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:16.091745 31314 external_mini_cluster.cc:1428] Setting key 728c6132cd1596d8cf096a7a91c54ca8
Jun 19 16:54:16 dist-test-slave-xb9n krb5kdc[2370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
I20260619 16:54:16.100811 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:16.091571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:53705 (local address 127.30.148.190:44677)
0619 16:54:16.091624 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:16.091626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:16.091829 (+ 203us) server_negotiation.cc:407] Connection header received
0619 16:54:16.092903 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:16.092904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:16.092916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:16.092990 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:16.093660 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:16.094214 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:16.095364 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:16.095688 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:16.095799 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:16.097873 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:16.097875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:16.097875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:16.097883 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:16.099514 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:16.099925 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:16.099926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:16.099926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:16.099990 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:16.100237 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:16.100238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:16.100242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:16.100345 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:16.100401 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:16.100636 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 16:54:16.100748 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 16:54:16.101006 2742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:16.091535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:44677 (local address 127.30.148.130:53705)
0619 16:54:16.091695 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:16.091733 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:16.092797 (+ 1064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:16.093042 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:16.093045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:16.093282 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:16.093532 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:16.093537 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:16.094304 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:16.094305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:16.095270 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:16.095273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:16.095813 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:16.095814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:16.095914 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:16.096418 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:16.096427 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:16.097782 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:16.099604 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:16.099610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:16.099612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:16.099813 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:16.100068 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:16.100069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:16.100069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:16.100164 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:16.100405 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:16.100409 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:16.100513 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 16:54:16.100610 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 16:54:16.100799 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260619 16:54:16.101491 2740 heartbeater.cc:347] Connected to a master server at 127.30.148.190:44677
I20260619 16:54:16.101645 2740 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:16.101811 2740 heartbeater.cc:511] Master 127.30.148.190:44677 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
I20260619 16:54:16.102229 2415 ts_manager.cc:194] Registered new tserver with Master: 98b436bbe4474293b9469373863b2f8e (127.30.148.130:45261)
I20260619 16:54:16.102595 2415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:53705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:54:16.115345 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:44677
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:35525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:16.196311 2745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:16.196521 2745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:16.196557 2745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:16.197762 2745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:16.197818 2745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:16.197861 2745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:16.199347 2745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:44677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:16.199610 2745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:16.199847 2745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:16.206168 2750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:16.206156 2751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:16.206275 2745 server_base.cc:1034] running on GCE node
W20260619 16:54:16.206146 2753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:16.206557 2745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:16.206825 2745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:16.207976 2745 hybrid_clock.cc:648] HybridClock initialized: now 1781888056207958 us; error 35 us; skew 500 ppm
Jun 19 16:54:16 dist-test-slave-xb9n krb5kdc[2370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:16.210891 2745 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:54:16.211347 2745 webserver.cc:460] Webserver started at http://127.30.148.131:46319/ using document root <none> and password file <none>
I20260619 16:54:16.211592 2745 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:16.211637 2745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:16.211773 2745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:16.212774 2745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "0ab95563114c403a9aadd22403a2f857"
format_stamp: "Formatted at 2026-06-19 16:54:16 on dist-test-slave-xb9n"
server_key: "40b029489da23eb3c7c5d1091cbf7f14"
server_key_iv: "c47b8c27f858e8cf8c8bc51fa5091ead"
server_key_version: "encryptionkey@0"
I20260619 16:54:16.213114 2745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "0ab95563114c403a9aadd22403a2f857"
format_stamp: "Formatted at 2026-06-19 16:54:16 on dist-test-slave-xb9n"
server_key: "40b029489da23eb3c7c5d1091cbf7f14"
server_key_iv: "c47b8c27f858e8cf8c8bc51fa5091ead"
server_key_version: "encryptionkey@0"
I20260619 16:54:16.214473 2745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:16.215234 2760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:16.215433 2745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:16.215519 2745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "0ab95563114c403a9aadd22403a2f857"
format_stamp: "Formatted at 2026-06-19 16:54:16 on dist-test-slave-xb9n"
server_key: "40b029489da23eb3c7c5d1091cbf7f14"
server_key_iv: "c47b8c27f858e8cf8c8bc51fa5091ead"
server_key_version: "encryptionkey@0"
I20260619 16:54:16.215607 2745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:16.230659 2745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:16.232931 2745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:16.233090 2745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:16.233311 2745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:16.233624 2745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:16.233680 2745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:16.233727 2745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:16.233774 2745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:16.239454 2745 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:41677
I20260619 16:54:16.239492 2873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:41677 every 8 connection(s)
I20260619 16:54:16.239831 2745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:54:16 dist-test-slave-xb9n krb5kdc[2370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:16.249190 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:16.240559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:36547 (local address 127.30.148.190:44677)
0619 16:54:16.240646 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:16.240649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:16.240805 (+ 156us) server_negotiation.cc:407] Connection header received
0619 16:54:16.241826 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:16.241827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:16.241840 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:16.241891 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:16.242510 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:16.243018 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:16.244025 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:16.244234 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:16.244317 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:16.246489 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:16.246491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:16.246491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:16.246497 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:16.247991 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:16.248381 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:16.248382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:16.248382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:16.248429 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:16.248643 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:16.248643 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:16.248644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:16.248745 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:16.248796 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:16.249071 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 16:54:16.249128 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 16:54:16.249372 2876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:16.240500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:44677 (local address 127.30.148.131:36547)
0619 16:54:16.240675 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:16.240717 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:16.241709 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:16.241959 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:16.241963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:16.242211 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:16.242413 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:16.242417 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:16.243127 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:16.243128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:16.243931 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:16.243934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:16.244326 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:16.244327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:16.244445 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:16.244955 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:16.244962 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:16.246408 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:16.248080 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:16.248086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:16.248087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:16.248312 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:16.248490 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:16.248491 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:16.248491 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:16.248588 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:16.248828 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:16.248833 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:16.248949 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 16:54:16.249055 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 16:54:16.249169 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260619 16:54:16.249620 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 2745
I20260619 16:54:16.249732 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:16.249946 31314 external_mini_cluster.cc:1428] Setting key 6a9a0362b7881499edeffb233695553e
I20260619 16:54:16.250111 2874 heartbeater.cc:347] Connected to a master server at 127.30.148.190:44677
I20260619 16:54:16.250205 2874 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:16.250391 2874 heartbeater.cc:511] Master 127.30.148.190:44677 requested a full tablet report, sending...
I20260619 16:54:16.250748 2415 ts_manager.cc:194] Registered new tserver with Master: 0ab95563114c403a9aadd22403a2f857 (127.30.148.131:41677)
I20260619 16:54:16.251102 2415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:36547
I20260619 16:54:16.261749 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 16:54:16 dist-test-slave-xb9n krb5kdc[2370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 16:54:16 dist-test-slave-xb9n krb5kdc[2370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.129@KRBTEST.COM
I20260619 16:54:16.283631 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:16.276335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42034 (local address 127.30.148.129:35021)
0619 16:54:16.276536 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:16.276540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:16.276586 (+ 46us) server_negotiation.cc:407] Connection header received
0619 16:54:16.276630 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:16.276632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:16.276647 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:16.276712 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:16.277226 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:16.277993 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:16.278682 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:16.278979 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:16.279035 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:16.281195 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:16.281201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:16.281201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:16.281207 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:16.282636 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:16.282947 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:16.282950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:16.282950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:16.283012 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:16.283165 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:16.283166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:16.283166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:16.283278 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:16.283372 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:16.283421 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 16:54:16.283475 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
Jun 19 16:54:16 dist-test-slave-xb9n krb5kdc[2370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:16.291208 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:16.284878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46998 (local address 127.30.148.190:44677)
0619 16:54:16.284942 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:16.284944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:16.284981 (+ 37us) server_negotiation.cc:407] Connection header received
0619 16:54:16.285059 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:16.285060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:16.285071 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:16.285120 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:16.285676 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:16.286149 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:16.286821 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:16.287023 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:16.287098 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:16.289138 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:16.289139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:16.289140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:16.289146 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:16.290310 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:16.290611 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:16.290612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:16.290613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:16.290661 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:16.290818 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:16.290819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:16.290819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:16.290942 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:16.291012 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:16.291090 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 16:54:16.291149 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 16:54:16.291447 2415 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46998
I20260619 16:54:16.291822 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 2477
I20260619 16:54:16.296950 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 2611
I20260619 16:54:16.301021 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 2745
I20260619 16:54:16.304711 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 2383
2026-06-19T16:54:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1804 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:16 dist-test-slave-xb9n krb5kdc[2886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:16 dist-test-slave-xb9n krb5kdc[2886](info): set up 2 sockets
Jun 19 16:54:16 dist-test-slave-xb9n krb5kdc[2886](info): commencing operation
krb5kdc: starting...
W20260619 16:54:17.333268 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:17 dist-test-slave-xb9n krb5kdc[2886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:54:17.439751 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:39213
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:39213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:17.516779 2899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:17.516958 2899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:17.516980 2899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:17.518126 2899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:17.518167 2899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:17.518181 2899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:17.518193 2899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:17.518205 2899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:17.519634 2899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:39213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:39213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:17.519866 2899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:17.520059 2899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:17.524611 2904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:17.524684 2907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:17.524740 2899 server_base.cc:1034] running on GCE node
W20260619 16:54:17.524612 2905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:17.525020 2899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:17.525290 2899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:17.526449 2899 hybrid_clock.cc:648] HybridClock initialized: now 1781888057526448 us; error 55 us; skew 500 ppm
Jun 19 16:54:17 dist-test-slave-xb9n krb5kdc[2886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888057, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:17.528771 2899 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:54:17.529119 2899 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:17.529186 2899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:17.529316 2899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:17.530326 2899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "3b06c282ec864cf1a54fe56d9aaf42ba"
format_stamp: "Formatted at 2026-06-19 16:54:17 on dist-test-slave-xb9n"
server_key: "aad96a7e712ba7952887925e9af36472"
server_key_iv: "8d7d5073d5b2a8c0aecae0560a1f3e94"
server_key_version: "encryptionkey@0"
I20260619 16:54:17.530678 2899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "3b06c282ec864cf1a54fe56d9aaf42ba"
format_stamp: "Formatted at 2026-06-19 16:54:17 on dist-test-slave-xb9n"
server_key: "aad96a7e712ba7952887925e9af36472"
server_key_iv: "8d7d5073d5b2a8c0aecae0560a1f3e94"
server_key_version: "encryptionkey@0"
I20260619 16:54:17.531885 2899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:17.532549 2913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:17.532716 2899 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:17.532809 2899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "3b06c282ec864cf1a54fe56d9aaf42ba"
format_stamp: "Formatted at 2026-06-19 16:54:17 on dist-test-slave-xb9n"
server_key: "aad96a7e712ba7952887925e9af36472"
server_key_iv: "8d7d5073d5b2a8c0aecae0560a1f3e94"
server_key_version: "encryptionkey@0"
I20260619 16:54:17.532889 2899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:17.543550 2899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:17.545212 2899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:17.545419 2899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:17.549259 2899 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:39213
I20260619 16:54:17.549286 2965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:39213 every 8 connection(s)
I20260619 16:54:17.549585 2899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:17.550415 2966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:17.552430 2966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba: Bootstrap starting.
I20260619 16:54:17.552927 2966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:17.553160 2966 log.cc:824] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:17.553689 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 2899
I20260619 16:54:17.553802 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:17.553856 2966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba: No bootstrap required, opened a new log
I20260619 16:54:17.553998 31314 external_mini_cluster.cc:1428] Setting key 80f340545b018dbf02adb874b0d94e58
I20260619 16:54:17.556625 2966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b06c282ec864cf1a54fe56d9aaf42ba" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39213 } }
I20260619 16:54:17.556746 2966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:17.556767 2966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b06c282ec864cf1a54fe56d9aaf42ba, State: Initialized, Role: FOLLOWER
I20260619 16:54:17.556887 2966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b06c282ec864cf1a54fe56d9aaf42ba" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39213 } }
I20260619 16:54:17.556958 2966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:17.557016 2966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:17.557063 2966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:17.557730 2966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b06c282ec864cf1a54fe56d9aaf42ba" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39213 } }
I20260619 16:54:17.557832 2966 leader_election.cc:302] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b06c282ec864cf1a54fe56d9aaf42ba; no voters:
I20260619 16:54:17.557965 2966 leader_election.cc:288] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:17.558023 2971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:17.558163 2971 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [term 1 LEADER]: Becoming Leader. State: Replica: 3b06c282ec864cf1a54fe56d9aaf42ba, State: Running, Role: LEADER
I20260619 16:54:17.558187 2966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:17.558302 2971 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b06c282ec864cf1a54fe56d9aaf42ba" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39213 } }
I20260619 16:54:17.558625 2973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b06c282ec864cf1a54fe56d9aaf42ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b06c282ec864cf1a54fe56d9aaf42ba" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39213 } } }
I20260619 16:54:17.558710 2973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:17.558962 2978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:17.559615 2978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:17.559952 2972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b06c282ec864cf1a54fe56d9aaf42ba. Latest consensus state: current_term: 1 leader_uuid: "3b06c282ec864cf1a54fe56d9aaf42ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b06c282ec864cf1a54fe56d9aaf42ba" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39213 } } }
I20260619 16:54:17.560122 2972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:17.561062 2978 catalog_manager.cc:1347] Generated new cluster ID: 9e813b0bea6c48b588c519b33e32a375
I20260619 16:54:17.561110 2978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:54:17 dist-test-slave-xb9n krb5kdc[2886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:17.565464 2970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:17.554473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33656 (local address 127.30.148.190:39213)
0619 16:54:17.554690 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:17.554697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:17.554733 (+ 36us) server_negotiation.cc:407] Connection header received
0619 16:54:17.555207 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:17.555213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:17.555254 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:17.555353 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:17.556139 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:17.557157 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:17.558272 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:17.558972 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:17.559048 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:17.561984 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:17.561987 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:17.561987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:17.561998 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:17.563926 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:17.564289 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:17.564291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:17.564292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:17.564393 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:17.564584 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:17.564585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:17.564585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:17.564868 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:17.565020 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:17.565100 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 16:54:17.565229 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":112,"threads_started":1}
I20260619 16:54:17.597868 2978 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:17.598385 2978 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:17.608410 2978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba: Generated new TSK 0
I20260619 16:54:17.608578 2978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:54:17.643906 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39213
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:17.731683 2992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:17.731868 2992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:17.731890 2992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:17.733019 2992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:17.733062 2992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:17.733115 2992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:17.735461 2992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:17.735713 2992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:17.735947 2992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:17.741042 2997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:17.741070 2998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:17.741088 2992 server_base.cc:1034] running on GCE node
W20260619 16:54:17.741098 3000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:17.741431 2992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:17.741711 2992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:17.742877 2992 hybrid_clock.cc:648] HybridClock initialized: now 1781888057742867 us; error 38 us; skew 500 ppm
Jun 19 16:54:17 dist-test-slave-xb9n krb5kdc[2886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888057, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:17.749691 2992 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:54:17.750043 2992 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:17.750113 2992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:17.750259 2992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:17.751274 2992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "dfa13a97500143f69eba6b459e684a3d"
format_stamp: "Formatted at 2026-06-19 16:54:17 on dist-test-slave-xb9n"
server_key: "e25a90c248c04d1bbc19b6517d19f05f"
server_key_iv: "7521372792f8d3f9d4aa6e306f646702"
server_key_version: "encryptionkey@0"
I20260619 16:54:17.751631 2992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "dfa13a97500143f69eba6b459e684a3d"
format_stamp: "Formatted at 2026-06-19 16:54:17 on dist-test-slave-xb9n"
server_key: "e25a90c248c04d1bbc19b6517d19f05f"
server_key_iv: "7521372792f8d3f9d4aa6e306f646702"
server_key_version: "encryptionkey@0"
I20260619 16:54:17.752898 2992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 16:54:17.753669 3006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:17.753844 2992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:17.753937 2992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "dfa13a97500143f69eba6b459e684a3d"
format_stamp: "Formatted at 2026-06-19 16:54:17 on dist-test-slave-xb9n"
server_key: "e25a90c248c04d1bbc19b6517d19f05f"
server_key_iv: "7521372792f8d3f9d4aa6e306f646702"
server_key_version: "encryptionkey@0"
I20260619 16:54:17.754012 2992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:17.767421 2992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:17.769438 2992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:17.769608 2992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:17.769831 2992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:17.770143 2992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:17.770197 2992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:17.770238 2992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:17.770267 2992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:17.776069 2992 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:42113
I20260619 16:54:17.776109 3119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:42113 every 8 connection(s)
I20260619 16:54:17.776356 2992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:17.778787 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 2992
I20260619 16:54:17.778900 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:17.779088 31314 external_mini_cluster.cc:1428] Setting key c870bae862ea673196339c7b5733da75
Jun 19 16:54:17 dist-test-slave-xb9n krb5kdc[2886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888057, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:17.786360 2970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:17.777007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:46333 (local address 127.30.148.190:39213)
0619 16:54:17.777049 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:17.777051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:17.777249 (+ 198us) server_negotiation.cc:407] Connection header received
0619 16:54:17.777759 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:17.777760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:17.777771 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:17.777821 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:17.778673 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:17.779447 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:17.780491 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:17.780833 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:17.780910 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:17.783438 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:17.783439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:17.783440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:17.783446 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:17.784878 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:17.785401 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:17.785402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:17.785403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:17.785461 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:17.785748 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:17.785749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:17.785749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:17.785851 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:17.785903 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:17.786234 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 16:54:17.786299 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:17.786638 3122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:17.776976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:39213 (local address 127.30.148.129:46333)
0619 16:54:17.777124 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:17.777159 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:17.777672 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:17.777873 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:17.777876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:17.778115 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:17.778584 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:17.778589 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:17.779538 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:17.779539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:17.780402 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:17.780405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:17.780934 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:17.780936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:17.781089 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:17.781641 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:17.781653 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:17.783341 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:17.784964 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:17.784971 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:17.784973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:17.785286 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:17.785533 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:17.785534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:17.785534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:17.785674 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:17.785920 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:17.785926 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:17.786095 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 16:54:17.786216 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 16:54:17.786380 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260619 16:54:17.787174 3120 heartbeater.cc:347] Connected to a master server at 127.30.148.190:39213
I20260619 16:54:17.787281 3120 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
I20260619 16:54:17.787464 3120 heartbeater.cc:511] Master 127.30.148.190:39213 requested a full tablet report, sending...
I20260619 16:54:17.788069 2930 ts_manager.cc:194] Registered new tserver with Master: dfa13a97500143f69eba6b459e684a3d (127.30.148.129:42113)
I20260619 16:54:17.788673 2930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:46333
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:54:17.801815 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39213
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:17.878926 3125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:17.879101 3125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:17.879124 3125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:17.880321 3125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:17.880358 3125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:17.880383 3125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:17.881822 3125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:17.882051 3125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:17.882265 3125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:17.886914 3131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:17.886919 3133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:17.886920 3130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:17.887104 3125 server_base.cc:1034] running on GCE node
I20260619 16:54:17.887351 3125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:17.887614 3125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:17.888754 3125 hybrid_clock.cc:648] HybridClock initialized: now 1781888057888746 us; error 26 us; skew 500 ppm
Jun 19 16:54:17 dist-test-slave-xb9n krb5kdc[2886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888057, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:17.891187 3125 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:54:17.891467 3125 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:17.891508 3125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:17.891592 3125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:17.892501 3125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd"
format_stamp: "Formatted at 2026-06-19 16:54:17 on dist-test-slave-xb9n"
server_key: "b1f92e3509df6c67aebec5323bdc0f28"
server_key_iv: "fdb7130051eaa6b45c9a9841c1793710"
server_key_version: "encryptionkey@0"
I20260619 16:54:17.892786 3125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd"
format_stamp: "Formatted at 2026-06-19 16:54:17 on dist-test-slave-xb9n"
server_key: "b1f92e3509df6c67aebec5323bdc0f28"
server_key_iv: "fdb7130051eaa6b45c9a9841c1793710"
server_key_version: "encryptionkey@0"
I20260619 16:54:17.893997 3125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:17.894610 3139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:17.894789 3125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:17.894872 3125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd"
format_stamp: "Formatted at 2026-06-19 16:54:17 on dist-test-slave-xb9n"
server_key: "b1f92e3509df6c67aebec5323bdc0f28"
server_key_iv: "fdb7130051eaa6b45c9a9841c1793710"
server_key_version: "encryptionkey@0"
I20260619 16:54:17.894939 3125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:17.910398 3125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:17.912495 3125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:17.912647 3125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:17.912842 3125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:17.913117 3125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:17.913147 3125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:17.913167 3125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:17.913216 3125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:17.918613 3125 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:38135
I20260619 16:54:17.918680 3252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:38135 every 8 connection(s)
I20260619 16:54:17.918917 3125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:54:17 dist-test-slave-xb9n krb5kdc[2886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888057, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:17.926004 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 3125
I20260619 16:54:17.926107 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:17.926290 31314 external_mini_cluster.cc:1428] Setting key 9bd3041f23f5464d8494ef1811f62502
I20260619 16:54:17.927568 3255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:17.919546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:39213 (local address 127.30.148.130:53579)
0619 16:54:17.919694 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:17.919730 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:17.920162 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:17.920346 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:17.920348 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:17.920571 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:17.921020 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:17.921025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:17.921713 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:17.921714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:17.922489 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:17.922490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:17.922810 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:17.922811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:17.922905 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:17.923271 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:17.923279 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:17.924544 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:17.926143 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:17.926150 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:17.926152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:17.926428 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:17.926702 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:17.926703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:17.926704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:17.926822 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:17.927112 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:17.927116 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:17.927234 (+ 118us) client_negotiation.cc:769] Sending connection context
0619 16:54:17.927313 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 16:54:17.927415 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260619 16:54:17.927596 2970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:17.919575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:53579 (local address 127.30.148.190:39213)
0619 16:54:17.919613 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:17.919615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:17.919796 (+ 181us) server_negotiation.cc:407] Connection header received
0619 16:54:17.920240 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:17.920240 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:17.920249 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:17.920299 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:17.921100 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:17.921639 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:17.922560 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:17.922745 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:17.922793 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:17.924624 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:17.924625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:17.924625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:17.924632 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:17.926040 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:17.926568 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:17.926569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:17.926570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:17.926627 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:17.926883 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:17.926884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:17.926884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:17.927044 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:17.927118 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:17.927360 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 16:54:17.927445 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:17.928623 3253 heartbeater.cc:347] Connected to a master server at 127.30.148.190:39213
I20260619 16:54:17.928741 3253 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:17.928872 3253 heartbeater.cc:511] Master 127.30.148.190:39213 requested a full tablet report, sending...
I20260619 16:54:17.929237 2930 ts_manager.cc:194] Registered new tserver with Master: f0f4a59d88324e80a5c7f718a2bcb4cd (127.30.148.130:38135)
I20260619 16:54:17.929672 2930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:53579
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:54:17.948993 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39213
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:18.030335 3258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:18.030570 3258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:18.030611 3258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:18.031749 3258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:18.031819 3258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:18.031864 3258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:18.033321 3258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:18.033613 3258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:18.033829 3258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:18.038551 3264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:18.038631 3266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:18.038556 3263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:18.038730 3258 server_base.cc:1034] running on GCE node
I20260619 16:54:18.038929 3258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:18.039157 3258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:18.040315 3258 hybrid_clock.cc:648] HybridClock initialized: now 1781888058040302 us; error 25 us; skew 500 ppm
Jun 19 16:54:18 dist-test-slave-xb9n krb5kdc[2886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888058, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:18.042762 3258 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:54:18.043133 3258 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:18.043200 3258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:18.043329 3258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:18.044278 3258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "1ac3c6e3bce943188c4bfd8e994eb411"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "397bf40f74fba5fdf05337caaa651005"
server_key_iv: "efba39ba707e8773b6f5923796005a32"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.044628 3258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "1ac3c6e3bce943188c4bfd8e994eb411"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "397bf40f74fba5fdf05337caaa651005"
server_key_iv: "efba39ba707e8773b6f5923796005a32"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.045944 3258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:18.046648 3272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.046805 3258 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:18.046896 3258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "1ac3c6e3bce943188c4bfd8e994eb411"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "397bf40f74fba5fdf05337caaa651005"
server_key_iv: "efba39ba707e8773b6f5923796005a32"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.046974 3258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:18.065740 3258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:18.067411 3258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:18.067585 3258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:18.067790 3258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:18.068071 3258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:18.068125 3258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.068163 3258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:18.068190 3258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.073670 3258 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:37621
I20260619 16:54:18.073745 3385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:37621 every 8 connection(s)
I20260619 16:54:18.073993 3258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:54:18 dist-test-slave-xb9n krb5kdc[2886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888058, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:18.082531 2970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.074679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:36753 (local address 127.30.148.190:39213)
0619 16:54:18.074753 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.074756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.074922 (+ 166us) server_negotiation.cc:407] Connection header received
0619 16:54:18.075364 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.075365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.075375 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.075420 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:18.076259 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.076724 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.077794 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.077990 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.078055 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.080136 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:18.080137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:18.080137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:18.080149 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:18.081340 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:18.081784 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:18.081785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:18.081785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:18.081834 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:18.082036 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:18.082037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:18.082037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:18.082130 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:18.082179 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.082398 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.082453 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 16:54:18.082677 3388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.074631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:39213 (local address 127.30.148.131:36753)
0619 16:54:18.074805 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.074839 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:18.075287 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:18.075478 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:18.075481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:18.075721 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:18.076183 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:18.076188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.076805 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.076806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:18.077714 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:18.077716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.078065 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.078066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:18.078155 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.078669 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:18.078678 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:18.080055 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:18.081492 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:18.081497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:18.081499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:18.081708 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:18.081891 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:18.081892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:18.081892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:18.081984 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:18.082182 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:18.082186 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:18.082294 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 16:54:18.082373 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 16:54:18.082479 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260619 16:54:18.083137 3386 heartbeater.cc:347] Connected to a master server at 127.30.148.190:39213
I20260619 16:54:18.083235 3386 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:18.083374 3386 heartbeater.cc:511] Master 127.30.148.190:39213 requested a full tablet report, sending...
I20260619 16:54:18.083591 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 3258
I20260619 16:54:18.083680 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:18.083778 2930 ts_manager.cc:194] Registered new tserver with Master: 1ac3c6e3bce943188c4bfd8e994eb411 (127.30.148.131:37621)
I20260619 16:54:18.083860 31314 external_mini_cluster.cc:1428] Setting key 1351de255ed18fd7da791de0804f3a2f
I20260619 16:54:18.084223 2930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:36753
I20260619 16:54:18.084633 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:54:18.091604 2970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.085910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33664 (local address 127.30.148.190:39213)
0619 16:54:18.085975 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.085976 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.086075 (+ 99us) server_negotiation.cc:407] Connection header received
0619 16:54:18.086147 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.086147 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.086156 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.086210 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:18.086727 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.087179 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.087911 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.088111 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.088193 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.089231 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:18.089232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:18.089233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:18.089240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:18.090435 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:18.090791 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:18.090792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:18.090792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:18.090841 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:18.091047 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:18.091048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:18.091048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:18.091184 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:18.091287 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.091420 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.091488 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 16:54:18.092981 2930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:18.094455 2930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:18.100792 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.096413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:42113 (local address 127.0.0.1:37520)
0619 16:54:18.096757 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.096799 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:18.096866 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:18.097102 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:18.097104 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:18.097107 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:18.097176 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:18.097177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.098609 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.098610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:18.099550 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:18.099552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.100432 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.100433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:18.100510 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.100536 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 16:54:18.100582 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 16:54:18.100632 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":81,"threads_started":1}
I20260619 16:54:18.100850 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.096744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54566 (local address 127.30.148.131:37621)
0619 16:54:18.096861 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.096866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.097316 (+ 450us) server_negotiation.cc:407] Connection header received
0619 16:54:18.097352 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.097388 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.097404 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.097478 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:18.097589 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.098350 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.099865 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.100319 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.100409 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.100461 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.100733 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.100770 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":62,"threads_started":1}
I20260619 16:54:18.100898 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.096617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.30.148.130:38135)
0619 16:54:18.096987 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.096991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.097000 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:54:18.097040 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.097042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.097074 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.097145 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:18.097348 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.098433 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.099856 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.100529 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.100601 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.100648 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.100784 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.100825 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":105,"threads_started":1}
I20260619 16:54:18.100943 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.096546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37520 (local address 127.30.148.129:42113)
0619 16:54:18.096907 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.096911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.096919 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:18.096980 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.096982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.096999 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.097086 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:18.097274 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.098465 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.099634 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.100313 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.100749 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.100805 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.100826 (+ 21us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.100863 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":76,"threads_started":1}
I20260619 16:54:18.101047 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.096648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:37621 (local address 127.0.0.1:54566)
0619 16:54:18.097197 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.097207 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:18.097256 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:18.097474 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:18.097475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:18.097477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:18.097517 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:18.097519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.098456 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.098457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:18.099769 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:18.099770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.100419 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.100419 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:18.100530 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.100540 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 16:54:18.100620 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 16:54:18.100661 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":41,"thread_start_us":84,"threads_started":1}
I20260619 16:54:18.101341 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.096452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:38135 (local address 127.0.0.1:40148)
0619 16:54:18.096792 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.096802 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:18.096866 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:18.097232 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:18.097233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:18.097235 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:18.097269 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:18.097270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.098524 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.098524 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:18.099767 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:18.099770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.100607 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.100608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:18.100661 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.100664 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:18.100716 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 16:54:18.100747 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":36,"thread_start_us":56,"threads_started":1}
I20260619 16:54:18.102092 3187 tablet_service.cc:1467] Processing CreateTablet for tablet 552f0f4ab0f5402da8e3c752f55c386a (DEFAULT_TABLE table=test-table [id=758fa17511fd4dc79907c8882f87280e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:18.102083 3320 tablet_service.cc:1467] Processing CreateTablet for tablet 552f0f4ab0f5402da8e3c752f55c386a (DEFAULT_TABLE table=test-table [id=758fa17511fd4dc79907c8882f87280e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:18.102145 3054 tablet_service.cc:1467] Processing CreateTablet for tablet 552f0f4ab0f5402da8e3c752f55c386a (DEFAULT_TABLE table=test-table [id=758fa17511fd4dc79907c8882f87280e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:18.102399 3054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 552f0f4ab0f5402da8e3c752f55c386a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:18.102399 3320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 552f0f4ab0f5402da8e3c752f55c386a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:18.102418 3187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 552f0f4ab0f5402da8e3c752f55c386a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:18.104800 3400 tablet_bootstrap.cc:492] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd: Bootstrap starting.
I20260619 16:54:18.105482 3400 tablet_bootstrap.cc:654] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:18.105561 3402 tablet_bootstrap.cc:492] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d: Bootstrap starting.
I20260619 16:54:18.106226 3402 tablet_bootstrap.cc:654] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:18.106321 3400 log.cc:824] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:18.106696 3401 tablet_bootstrap.cc:492] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411: Bootstrap starting.
I20260619 16:54:18.106853 3402 log.cc:824] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:18.107337 3401 tablet_bootstrap.cc:654] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:18.107649 3402 tablet_bootstrap.cc:492] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d: No bootstrap required, opened a new log
I20260619 16:54:18.107736 3402 ts_tablet_manager.cc:1397] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:54:18.107957 3401 log.cc:824] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:18.108469 3400 tablet_bootstrap.cc:492] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd: No bootstrap required, opened a new log
I20260619 16:54:18.108538 3400 ts_tablet_manager.cc:1397] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260619 16:54:18.108628 3401 tablet_bootstrap.cc:492] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411: No bootstrap required, opened a new log
I20260619 16:54:18.108700 3401 ts_tablet_manager.cc:1397] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:54:18.109812 3402 raft_consensus.cc:348] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa13a97500143f69eba6b459e684a3d" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42113 } } peers { permanent_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38135 } } peers { permanent_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37621 } }
I20260619 16:54:18.109977 3402 raft_consensus.cc:374] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:18.110013 3402 raft_consensus.cc:729] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfa13a97500143f69eba6b459e684a3d, State: Initialized, Role: FOLLOWER
I20260619 16:54:18.110163 3402 consensus_queue.cc:260] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa13a97500143f69eba6b459e684a3d" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42113 } } peers { permanent_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38135 } } peers { permanent_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37621 } }
I20260619 16:54:18.110219 3401 raft_consensus.cc:348] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa13a97500143f69eba6b459e684a3d" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42113 } } peers { permanent_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38135 } } peers { permanent_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37621 } }
I20260619 16:54:18.110383 3401 raft_consensus.cc:374] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:18.110427 3401 raft_consensus.cc:729] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ac3c6e3bce943188c4bfd8e994eb411, State: Initialized, Role: FOLLOWER
I20260619 16:54:18.110437 3402 ts_tablet_manager.cc:1428] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:54:18.110585 3120 heartbeater.cc:503] Master 127.30.148.190:39213 was elected leader, sending a full tablet report...
I20260619 16:54:18.110559 3401 consensus_queue.cc:260] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa13a97500143f69eba6b459e684a3d" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42113 } } peers { permanent_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38135 } } peers { permanent_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37621 } }
I20260619 16:54:18.110831 3386 heartbeater.cc:503] Master 127.30.148.190:39213 was elected leader, sending a full tablet report...
I20260619 16:54:18.110842 3400 raft_consensus.cc:348] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa13a97500143f69eba6b459e684a3d" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42113 } } peers { permanent_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38135 } } peers { permanent_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37621 } }
I20260619 16:54:18.110991 3400 raft_consensus.cc:374] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:18.111101 3400 raft_consensus.cc:729] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0f4a59d88324e80a5c7f718a2bcb4cd, State: Initialized, Role: FOLLOWER
I20260619 16:54:18.110831 3401 ts_tablet_manager.cc:1428] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 16:54:18.111245 3400 consensus_queue.cc:260] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa13a97500143f69eba6b459e684a3d" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42113 } } peers { permanent_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38135 } } peers { permanent_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37621 } }
I20260619 16:54:18.111543 3400 ts_tablet_manager.cc:1428] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 16:54:18.111581 3253 heartbeater.cc:503] Master 127.30.148.190:39213 was elected leader, sending a full tablet report...
W20260619 16:54:18.169445 3254 tablet.cc:2369] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:54:18.276993 3121 tablet.cc:2369] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:54:18.316738 3407 raft_consensus.cc:482] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:18.316849 3407 raft_consensus.cc:504] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa13a97500143f69eba6b459e684a3d" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42113 } } peers { permanent_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38135 } } peers { permanent_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37621 } }
I20260619 16:54:18.317189 3407 leader_election.cc:288] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfa13a97500143f69eba6b459e684a3d (127.30.148.129:42113), f0f4a59d88324e80a5c7f718a2bcb4cd (127.30.148.130:38135)
I20260619 16:54:18.321084 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.317641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:48369 (local address 127.30.148.130:38135)
0619 16:54:18.317687 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.317689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.317697 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:18.317736 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.317737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.317746 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.317795 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:18.317958 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.318698 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.320168 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.320735 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.320798 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.320807 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.320978 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.321014 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 16:54:18.321084 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.317478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:58797 (local address 127.30.148.129:42113)
0619 16:54:18.317543 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.317545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.317561 (+ 16us) server_negotiation.cc:407] Connection header received
0619 16:54:18.317676 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.317676 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.317687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.317762 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:18.317958 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.318698 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.320168 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.320694 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.320798 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.320807 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.320979 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.321014 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 16:54:18.321127 3388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.317401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:42113 (local address 127.30.148.131:58797)
0619 16:54:18.317519 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.317529 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:18.317593 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:18.317791 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:18.317792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:18.317794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:18.317878 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:18.317880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.318779 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.318780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:18.320075 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:18.320078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.320807 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.320808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:18.320893 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.320897 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:18.320954 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 16:54:18.320998 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":99}
I20260619 16:54:18.321133 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.317408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:38135 (local address 127.30.148.131:48369)
0619 16:54:18.317626 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.317636 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:18.317676 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:18.317804 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:18.317805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:18.317818 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:18.317878 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:18.317880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.318779 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.318780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:18.320075 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:18.320078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.320807 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.320808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:18.320893 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.320897 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:18.320954 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 16:54:18.320995 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":51,"thread_start_us":135,"threads_started":1}
I20260619 16:54:18.321523 3074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "552f0f4ab0f5402da8e3c752f55c386a" candidate_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfa13a97500143f69eba6b459e684a3d" is_pre_election: true
I20260619 16:54:18.321545 3207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "552f0f4ab0f5402da8e3c752f55c386a" candidate_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd" is_pre_election: true
I20260619 16:54:18.321681 3074 raft_consensus.cc:2454] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ac3c6e3bce943188c4bfd8e994eb411 in term 0.
I20260619 16:54:18.321728 3207 raft_consensus.cc:2454] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ac3c6e3bce943188c4bfd8e994eb411 in term 0.
I20260619 16:54:18.321933 3276 leader_election.cc:302] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ac3c6e3bce943188c4bfd8e994eb411, f0f4a59d88324e80a5c7f718a2bcb4cd; no voters:
I20260619 16:54:18.322045 3407 raft_consensus.cc:2781] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:18.322113 3407 raft_consensus.cc:482] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:18.322149 3407 raft_consensus.cc:3037] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:18.322816 3407 raft_consensus.cc:504] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa13a97500143f69eba6b459e684a3d" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42113 } } peers { permanent_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38135 } } peers { permanent_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37621 } }
I20260619 16:54:18.322963 3407 leader_election.cc:288] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [CANDIDATE]: Term 1 election: Requested vote from peers dfa13a97500143f69eba6b459e684a3d (127.30.148.129:42113), f0f4a59d88324e80a5c7f718a2bcb4cd (127.30.148.130:38135)
I20260619 16:54:18.323136 3207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "552f0f4ab0f5402da8e3c752f55c386a" candidate_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd"
I20260619 16:54:18.323138 3074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "552f0f4ab0f5402da8e3c752f55c386a" candidate_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfa13a97500143f69eba6b459e684a3d"
I20260619 16:54:18.323239 3207 raft_consensus.cc:3037] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:18.323238 3074 raft_consensus.cc:3037] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:18.323917 3207 raft_consensus.cc:2454] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ac3c6e3bce943188c4bfd8e994eb411 in term 1.
I20260619 16:54:18.323917 3074 raft_consensus.cc:2454] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ac3c6e3bce943188c4bfd8e994eb411 in term 1.
I20260619 16:54:18.324116 3275 leader_election.cc:302] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ac3c6e3bce943188c4bfd8e994eb411, dfa13a97500143f69eba6b459e684a3d; no voters:
I20260619 16:54:18.324214 3407 raft_consensus.cc:2781] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:18.324357 3407 raft_consensus.cc:686] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 1 LEADER]: Becoming Leader. State: Replica: 1ac3c6e3bce943188c4bfd8e994eb411, State: Running, Role: LEADER
W20260619 16:54:18.324453 3387 tablet.cc:2369] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:54:18.324554 3407 consensus_queue.cc:237] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa13a97500143f69eba6b459e684a3d" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42113 } } peers { permanent_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38135 } } peers { permanent_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37621 } }
I20260619 16:54:18.325318 2928 catalog_manager.cc:5495] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ac3c6e3bce943188c4bfd8e994eb411 (127.30.148.131). New cstate: current_term: 1 leader_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfa13a97500143f69eba6b459e684a3d" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ac3c6e3bce943188c4bfd8e994eb411" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37621 } health_report { overall_health: HEALTHY } } }
I20260619 16:54:18.384594 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.382513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54578 (local address 127.30.148.131:37621)
0619 16:54:18.382571 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.382573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.382589 (+ 16us) server_negotiation.cc:407] Connection header received
0619 16:54:18.382622 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.382623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.382638 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.382695 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:18.382819 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.383275 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.383952 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:18.384125 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:18.384172 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:18.384298 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:18.384387 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:18.384444 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.384474 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.384488 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 16:54:18.386529 3207 raft_consensus.cc:1261] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd [term 1 FOLLOWER]: Refusing update from remote peer 1ac3c6e3bce943188c4bfd8e994eb411: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:18.386551 3074 raft_consensus.cc:1261] T 552f0f4ab0f5402da8e3c752f55c386a P dfa13a97500143f69eba6b459e684a3d [term 1 FOLLOWER]: Refusing update from remote peer 1ac3c6e3bce943188c4bfd8e994eb411: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:18.386883 3410 consensus_queue.cc:1035] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0f4a59d88324e80a5c7f718a2bcb4cd" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:18.387040 3407 consensus_queue.cc:1035] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfa13a97500143f69eba6b459e684a3d" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:18.389148 3413 mvcc.cc:205] Tried to move back new op lower bound from 7298613487148191744 to 7298613486898044928. Current Snapshot: MvccSnapshot[applied={T|T < 7298613487148191744}]
I20260619 16:54:18.390311 3414 mvcc.cc:205] Tried to move back new op lower bound from 7298613487148191744 to 7298613486898044928. Current Snapshot: MvccSnapshot[applied={T|T < 7298613487148191744}]
I20260619 16:54:18.391535 2928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:18.391613 2928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:18.392378 2928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b06c282ec864cf1a54fe56d9aaf42ba: Sending DeleteTablet for 3 replicas of tablet 552f0f4ab0f5402da8e3c752f55c386a
I20260619 16:54:18.392696 3054 tablet_service.cc:1514] Processing DeleteTablet for tablet 552f0f4ab0f5402da8e3c752f55c386a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:18 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:37520
I20260619 16:54:18.392750 3320 tablet_service.cc:1514] Processing DeleteTablet for tablet 552f0f4ab0f5402da8e3c752f55c386a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:18 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:54566
I20260619 16:54:18.392915 3423 tablet_replica.cc:330] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411: stopping tablet replica
I20260619 16:54:18.392913 3187 tablet_service.cc:1514] Processing DeleteTablet for tablet 552f0f4ab0f5402da8e3c752f55c386a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:18 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:40148
I20260619 16:54:18.392931 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 2992
I20260619 16:54:18.393692 3423 raft_consensus.cc:2229] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:18.393798 3423 raft_consensus.cc:2258] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:18.393944 3424 tablet_replica.cc:330] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd: stopping tablet replica
I20260619 16:54:18.394093 3424 raft_consensus.cc:2229] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:18.394235 3424 raft_consensus.cc:2258] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:18.394248 3423 ts_tablet_manager.cc:1905] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:18.394762 3424 ts_tablet_manager.cc:1905] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:18.395983 3423 ts_tablet_manager.cc:1918] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:18.396112 3423 log.cc:1196] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/552f0f4ab0f5402da8e3c752f55c386a
I20260619 16:54:18.396447 3423 ts_tablet_manager.cc:1939] T 552f0f4ab0f5402da8e3c752f55c386a P 1ac3c6e3bce943188c4bfd8e994eb411: Deleting consensus metadata
I20260619 16:54:18.396451 3424 ts_tablet_manager.cc:1918] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:18.396505 3424 log.cc:1196] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/552f0f4ab0f5402da8e3c752f55c386a
I20260619 16:54:18.396767 3424 ts_tablet_manager.cc:1939] T 552f0f4ab0f5402da8e3c752f55c386a P f0f4a59d88324e80a5c7f718a2bcb4cd: Deleting consensus metadata
I20260619 16:54:18.396946 2916 catalog_manager.cc:4841] TS 1ac3c6e3bce943188c4bfd8e994eb411 (127.30.148.131:37621): tablet 552f0f4ab0f5402da8e3c752f55c386a (table test-table [id=758fa17511fd4dc79907c8882f87280e]) successfully deleted
I20260619 16:54:18.397217 2917 catalog_manager.cc:4841] TS f0f4a59d88324e80a5c7f718a2bcb4cd (127.30.148.130:38135): tablet 552f0f4ab0f5402da8e3c752f55c386a (table test-table [id=758fa17511fd4dc79907c8882f87280e]) successfully deleted
W20260619 16:54:18.400688 2916 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:42113 (error 108)
I20260619 16:54:18.400864 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 3125
I20260619 16:54:18.401458 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.400970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:42113 (local address 127.0.0.1:37524)
0619 16:54:18.401392 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.401433 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:42113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371}
W20260619 16:54:18.405500 2916 catalog_manager.cc:4584] TS dfa13a97500143f69eba6b459e684a3d (127.30.148.129:42113): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 552f0f4ab0f5402da8e3c752f55c386a: Network error: Client connection negotiation failed: client connection to 127.30.148.129:42113: connect: Connection refused (error 111)
I20260619 16:54:18.405933 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 3258
I20260619 16:54:18.410038 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 2899
2026-06-19T16:54:18Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2105 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T16:54:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:18Z Disabled control of system clock
I20260619 16:54:18.435868 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:45861
--webserver_interface=127.30.148.190
--webserver_port=0
--builtin_ntp_servers=127.30.148.148:35631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:45861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 16:54:18.514168 3429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:18.514348 3429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:18.514371 3429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:18.515504 3429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:18.515538 3429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:18.515550 3429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:18.515563 3429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:18.515576 3429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:18.516863 3429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:45861
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:45861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:18.517050 3429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:18.517266 3429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:54:18.521967 3429 server_base.cc:1034] running on GCE node
W20260619 16:54:18.521938 3435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:18.521929 3437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:18.521940 3434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:18.522382 3429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:18.522656 3429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:18.523794 3429 hybrid_clock.cc:648] HybridClock initialized: now 1781888058523776 us; error 39 us; skew 500 ppm
I20260619 16:54:18.524933 3429 webserver.cc:460] Webserver started at http://127.30.148.190:40883/ using document root <none> and password file <none>
I20260619 16:54:18.525177 3429 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:18.525223 3429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:18.525309 3429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:18.526304 3429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "0de2c4d2df2b4ee5b1cdd599d4ab8b5d"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "8959e7aea4bc055f6de160f5a56647f3"
server_key_iv: "56a062d0f94056792e485661cd01227a"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.526587 3429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "0de2c4d2df2b4ee5b1cdd599d4ab8b5d"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "8959e7aea4bc055f6de160f5a56647f3"
server_key_iv: "56a062d0f94056792e485661cd01227a"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.527766 3429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:18.528404 3443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.528671 3429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:18.528751 3429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "0de2c4d2df2b4ee5b1cdd599d4ab8b5d"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "8959e7aea4bc055f6de160f5a56647f3"
server_key_iv: "56a062d0f94056792e485661cd01227a"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.528802 3429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:18.531898 3429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:18.532104 3429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:18.532195 3429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:18.535665 3429 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:45861
I20260619 16:54:18.535734 3495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:45861 every 8 connection(s)
I20260619 16:54:18.536041 3429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:18.536679 3496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:18.539314 3496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d: Bootstrap starting.
I20260619 16:54:18.539359 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 3429
I20260619 16:54:18.539467 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:18.539676 31314 external_mini_cluster.cc:1428] Setting key a373cd848e962f7547cb4adf8f4c6dd9
I20260619 16:54:18.539934 3496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:18.540292 3496 log.cc:824] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:18.540983 3496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d: No bootstrap required, opened a new log
I20260619 16:54:18.542141 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.540190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40738 (local address 127.30.148.190:45861)
0619 16:54:18.540375 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.540382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.540432 (+ 50us) server_negotiation.cc:407] Connection header received
0619 16:54:18.540814 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.540820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.540854 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.541010 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:18.541735 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:18.541738 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:18.541738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:18.541742 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:18.541791 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:18.541801 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:18.541891 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.541927 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.541980 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260619 16:54:18.542891 3496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de2c4d2df2b4ee5b1cdd599d4ab8b5d" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 45861 } }
I20260619 16:54:18.543049 3496 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:18.543094 3496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de2c4d2df2b4ee5b1cdd599d4ab8b5d, State: Initialized, Role: FOLLOWER
I20260619 16:54:18.543208 3496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de2c4d2df2b4ee5b1cdd599d4ab8b5d" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 45861 } }
I20260619 16:54:18.543303 3496 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:18.543347 3496 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:18.543394 3496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:18.544301 3496 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de2c4d2df2b4ee5b1cdd599d4ab8b5d" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 45861 } }
I20260619 16:54:18.544435 3496 leader_election.cc:302] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0de2c4d2df2b4ee5b1cdd599d4ab8b5d; no voters:
I20260619 16:54:18.544587 3496 leader_election.cc:288] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:18.544610 3501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:18.544740 3501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [term 1 LEADER]: Becoming Leader. State: Replica: 0de2c4d2df2b4ee5b1cdd599d4ab8b5d, State: Running, Role: LEADER
I20260619 16:54:18.544849 3496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:18.544873 3501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de2c4d2df2b4ee5b1cdd599d4ab8b5d" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 45861 } }
I20260619 16:54:18.545275 3502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0de2c4d2df2b4ee5b1cdd599d4ab8b5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de2c4d2df2b4ee5b1cdd599d4ab8b5d" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 45861 } } }
I20260619 16:54:18.545281 3503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0de2c4d2df2b4ee5b1cdd599d4ab8b5d. Latest consensus state: current_term: 1 leader_uuid: "0de2c4d2df2b4ee5b1cdd599d4ab8b5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de2c4d2df2b4ee5b1cdd599d4ab8b5d" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 45861 } } }
I20260619 16:54:18.545514 3503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:18.545426 3502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d [sys.catalog]: This master's current role is: LEADER
W20260619 16:54:18.546566 3517 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:54:18.546618 3517 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:54:18.546667 3510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:18.546792 3510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:18.548141 3510 catalog_manager.cc:1347] Generated new cluster ID: 99f547480774468d80ebac6b0867d1d9
I20260619 16:54:18.548200 3510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:18.551321 3510 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:18.551487 3510 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:18.554489 3510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d: Generated new TSK 0
I20260619 16:54:18.554654 3510 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:54:18.559598 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:45861
--builtin_ntp_servers=127.30.148.148:35631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 16:54:18.643376 3520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:18.643666 3520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:18.643705 3520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:18.645587 3520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:18.645640 3520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:18.645677 3520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:18.647706 3520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:45861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:18.648092 3520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:18.648414 3520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:18.653406 3526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:18.653456 3525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:18.654109 3520 server_base.cc:1034] running on GCE node
W20260619 16:54:18.657424 3528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:18.657675 3520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:18.657958 3520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:18.659116 3520 hybrid_clock.cc:648] HybridClock initialized: now 1781888058659107 us; error 29 us; skew 500 ppm
I20260619 16:54:18.660418 3520 webserver.cc:460] Webserver started at http://127.30.148.129:37777/ using document root <none> and password file <none>
I20260619 16:54:18.660678 3520 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:18.660723 3520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:18.660851 3520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:18.661943 3520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "d4042405fbf64e2b8c22f337bc823b00"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "06234b3da0c411bdbcd467eaecd65c8b"
server_key_iv: "29e4d16ab56f0c13003957454657bf48"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.662293 3520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "d4042405fbf64e2b8c22f337bc823b00"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "06234b3da0c411bdbcd467eaecd65c8b"
server_key_iv: "29e4d16ab56f0c13003957454657bf48"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.663625 3520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:18.664366 3534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.664543 3520 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:18.664644 3520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "d4042405fbf64e2b8c22f337bc823b00"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "06234b3da0c411bdbcd467eaecd65c8b"
server_key_iv: "29e4d16ab56f0c13003957454657bf48"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.664716 3520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:18.668032 3520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:18.668290 3520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:18.668419 3520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:18.668653 3520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:18.669013 3520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:18.669049 3520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.669067 3520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:18.669111 3520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.675294 3520 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:45321
I20260619 16:54:18.675339 3647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:45321 every 8 connection(s)
I20260619 16:54:18.675630 3520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:18.678045 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 3520
I20260619 16:54:18.678144 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:18.678376 31314 external_mini_cluster.cc:1428] Setting key 2c0961178aee3b9796fe4dc0c6fc76a1
I20260619 16:54:18.678706 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.676228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:47549 (local address 127.30.148.190:45861)
0619 16:54:18.676269 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.676271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.676475 (+ 204us) server_negotiation.cc:407] Connection header received
0619 16:54:18.676919 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.676920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.676924 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.676980 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:18.678389 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:18.678391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:18.678391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:18.678396 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:18.678417 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:18.678422 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:18.678498 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.678615 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.678631 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:18.678890 3650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.676225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:45861 (local address 127.30.148.129:47549)
0619 16:54:18.676377 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.676407 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:18.676810 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:18.677058 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:18.677061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:18.677071 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:18.678246 (+ 1175us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:18.678246 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:18.678280 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:18.678305 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:18.678496 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:18.678499 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:18.678501 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:54:18.678622 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 16:54:18.678703 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260619 16:54:18.679301 3648 heartbeater.cc:347] Connected to a master server at 127.30.148.190:45861
I20260619 16:54:18.679394 3648 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:18.679409 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:45861
--builtin_ntp_servers=127.30.148.148:35631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 16:54:18.679541 3648 heartbeater.cc:511] Master 127.30.148.190:45861 requested a full tablet report, sending...
I20260619 16:54:18.680009 3460 ts_manager.cc:194] Registered new tserver with Master: d4042405fbf64e2b8c22f337bc823b00 (127.30.148.129:45321)
W20260619 16:54:18.758991 3651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:18.759161 3651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:18.759182 3651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:18.760212 3651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:18.760246 3651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:18.760270 3651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:18.761560 3651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:45861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:18.761735 3651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:18.761945 3651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:18.766551 3656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:18.766613 3659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:18.766616 3651 server_base.cc:1034] running on GCE node
W20260619 16:54:18.766538 3657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:18.766945 3651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:18.767210 3651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:18.768388 3651 hybrid_clock.cc:648] HybridClock initialized: now 1781888058768362 us; error 40 us; skew 500 ppm
I20260619 16:54:18.769549 3651 webserver.cc:460] Webserver started at http://127.30.148.130:46325/ using document root <none> and password file <none>
I20260619 16:54:18.769793 3651 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:18.769857 3651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:18.769984 3651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:18.770972 3651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "929af2f81b459088f66aa7a3156a4aac"
server_key_iv: "323313519130b673585f6728c2273eb1"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.771319 3651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "929af2f81b459088f66aa7a3156a4aac"
server_key_iv: "323313519130b673585f6728c2273eb1"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.772629 3651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:18.773331 3665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.773504 3651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:18.773597 3651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "929af2f81b459088f66aa7a3156a4aac"
server_key_iv: "323313519130b673585f6728c2273eb1"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.773676 3651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:18.776836 3651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:18.777082 3651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:18.777211 3651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:18.777463 3651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:18.777758 3651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:18.777814 3651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.777851 3651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:18.777880 3651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.783556 3651 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:35653
I20260619 16:54:18.783591 3778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:35653 every 8 connection(s)
I20260619 16:54:18.783893 3651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:54:18.786480 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.784570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:50449 (local address 127.30.148.190:45861)
0619 16:54:18.784648 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.784651 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.784799 (+ 148us) server_negotiation.cc:407] Connection header received
0619 16:54:18.785186 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.785187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.785192 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.785249 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:18.786300 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:18.786301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:18.786301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:18.786305 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:18.786319 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:18.786323 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:18.786374 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.786419 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.786429 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 16:54:18.786645 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.784535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:45861 (local address 127.30.148.130:50449)
0619 16:54:18.784695 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.784724 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:18.785114 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:18.785297 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:18.785300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:18.785307 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:18.786175 (+ 868us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:18.786175 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:18.786199 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:18.786224 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:18.786374 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:18.786376 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:18.786378 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:54:18.786425 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 16:54:18.786468 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":88,"threads_started":1}
I20260619 16:54:18.787156 3779 heartbeater.cc:347] Connected to a master server at 127.30.148.190:45861
I20260619 16:54:18.787226 3779 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:18.787321 3779 heartbeater.cc:511] Master 127.30.148.190:45861 requested a full tablet report, sending...
I20260619 16:54:18.787633 3460 ts_manager.cc:194] Registered new tserver with Master: 89e6c93ae4c64f1cba1bdbdadf9b6ad4 (127.30.148.130:35653)
I20260619 16:54:18.794090 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 3651
I20260619 16:54:18.794169 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:18.794296 31314 external_mini_cluster.cc:1428] Setting key b8b0d8d2316fbaa2dc408d893f406086
I20260619 16:54:18.795101 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:45861
--builtin_ntp_servers=127.30.148.148:35631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 16:54:18.872237 3782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:18.872452 3782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:18.872489 3782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:18.873654 3782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:18.873720 3782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:18.873764 3782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:18.875077 3782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:45861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:18.875357 3782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:18.875628 3782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:18.880494 3790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:18.880505 3787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:18.880524 3788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:18.880664 3782 server_base.cc:1034] running on GCE node
I20260619 16:54:18.880901 3782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:18.881132 3782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:18.882298 3782 hybrid_clock.cc:648] HybridClock initialized: now 1781888058882278 us; error 58 us; skew 500 ppm
I20260619 16:54:18.883486 3782 webserver.cc:460] Webserver started at http://127.30.148.131:36573/ using document root <none> and password file <none>
I20260619 16:54:18.883711 3782 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:18.883778 3782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:18.883908 3782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:18.884902 3782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "9902ef351c584921ba333472736b0188"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "72ae9d9ef3ad2dd93b7aed5dcec6082e"
server_key_iv: "d32f68e20c6b5aa3b69dba237eea92fa"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.885260 3782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "9902ef351c584921ba333472736b0188"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "72ae9d9ef3ad2dd93b7aed5dcec6082e"
server_key_iv: "d32f68e20c6b5aa3b69dba237eea92fa"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.886508 3782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:18.887169 3796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.887355 3782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:18.887447 3782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "9902ef351c584921ba333472736b0188"
format_stamp: "Formatted at 2026-06-19 16:54:18 on dist-test-slave-xb9n"
server_key: "72ae9d9ef3ad2dd93b7aed5dcec6082e"
server_key_iv: "d32f68e20c6b5aa3b69dba237eea92fa"
server_key_version: "encryptionkey@0"
I20260619 16:54:18.887547 3782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:18.890776 3782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:18.891021 3782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:18.891150 3782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:18.891348 3782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:18.891629 3782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:18.891680 3782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.891717 3782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:18.891745 3782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:18.897050 3782 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:39757
I20260619 16:54:18.897083 3909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:39757 every 8 connection(s)
I20260619 16:54:18.897416 3782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:54:18.899070 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 3782
I20260619 16:54:18.899183 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:18.899348 31314 external_mini_cluster.cc:1428] Setting key 5884b7b4d98707f31150c777e4ec2204
I20260619 16:54:18.900303 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.898106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:51837 (local address 127.30.148.190:45861)
0619 16:54:18.898157 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.898159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.898375 (+ 216us) server_negotiation.cc:407] Connection header received
0619 16:54:18.898760 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.898761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.898766 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.898807 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:18.900098 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:18.900099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:18.900099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:18.900103 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:18.900119 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:18.900123 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:18.900174 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.900243 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.900253 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:54:18.900521 3912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.898093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:45861 (local address 127.30.148.131:51837)
0619 16:54:18.898265 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.898293 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:18.898672 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:18.898855 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:18.898857 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:18.898864 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:18.899946 (+ 1082us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:18.899947 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:18.899979 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:18.900012 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:18.900172 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:18.900176 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:18.900181 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:54:18.900248 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 16:54:18.900291 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260619 16:54:18.900944 3910 heartbeater.cc:347] Connected to a master server at 127.30.148.190:45861
I20260619 16:54:18.901033 3910 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:18.901167 3910 heartbeater.cc:511] Master 127.30.148.190:45861 requested a full tablet report, sending...
I20260619 16:54:18.901520 3460 ts_manager.cc:194] Registered new tserver with Master: 9902ef351c584921ba333472736b0188 (127.30.148.131:39757)
I20260619 16:54:18.910693 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:54:18.912848 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.911964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40744 (local address 127.30.148.190:45861)
0619 16:54:18.912034 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.912036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.912071 (+ 35us) server_negotiation.cc:407] Connection header received
0619 16:54:18.912122 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.912123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.912127 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.912175 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:18.912637 (+ 462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:18.912637 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:18.912638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:18.912641 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:18.912651 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:18.912655 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:18.912701 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.912734 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.912744 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 16:54:18.914269 3460 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:18.915743 3460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:18.919385 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.917617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:39757 (local address 127.0.0.1:36636)
0619 16:54:18.917793 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.917828 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:18.917878 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:18.918347 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:18.918348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:18.918350 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:18.918941 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:18.918942 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:18.918949 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:18.918960 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:18.919184 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:18.919191 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:18.919193 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:54:18.919293 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 16:54:18.919307 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":80,"threads_started":1}
I20260619 16:54:18.919425 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.917650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36636 (local address 127.30.148.131:39757)
0619 16:54:18.918229 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.918232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.918239 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:54:18.918256 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.918257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.918264 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.918345 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:18.919040 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:18.919047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:18.919047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:18.919051 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:18.919096 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:18.919115 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:18.919175 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.919314 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.919330 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":67,"threads_started":1}
I20260619 16:54:18.919759 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.917666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:45321 (local address 127.0.0.1:49568)
0619 16:54:18.918675 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.918686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:18.918732 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:18.918920 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:18.918922 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:18.918926 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:18.919345 (+ 419us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:18.919345 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:18.919351 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:18.919361 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:18.919591 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:18.919592 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:18.919592 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 16:54:18.919665 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 16:54:18.919677 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"thread_start_us":68,"threads_started":1}
I20260619 16:54:18.919781 3923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.917865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49568 (local address 127.30.148.129:45321)
0619 16:54:18.918045 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.918051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.918792 (+ 741us) server_negotiation.cc:407] Connection header received
0619 16:54:18.918833 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.918835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.918846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.918908 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:18.919441 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:18.919448 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:18.919448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:18.919452 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:18.919493 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:18.919515 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:18.919568 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.919674 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.919688 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":75,"threads_started":1}
I20260619 16:54:18.919931 3918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.917618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:35653 (local address 127.0.0.1:47624)
0619 16:54:18.918098 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:18.918106 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:18.918151 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:18.918298 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:18.918303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:18.918309 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:18.918782 (+ 473us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:18.918783 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:18.918798 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:18.918821 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:18.919809 (+ 988us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:18.919810 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:18.919810 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 16:54:18.919857 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 16:54:18.919871 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260619 16:54:18.920154 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:18.917839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47624 (local address 127.30.148.130:35653)
0619 16:54:18.918022 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:18.918025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:18.918144 (+ 119us) server_negotiation.cc:407] Connection header received
0619 16:54:18.918198 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:18.918200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:18.918207 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:18.918283 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:18.919642 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:18.919645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:18.919645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:18.919648 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:18.919673 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:18.919685 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:18.919782 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:18.920030 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 16:54:18.920051 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
I20260619 16:54:18.920405 3582 tablet_service.cc:1467] Processing CreateTablet for tablet 75b2217dedc247fab9df91297e4c23a6 (DEFAULT_TABLE table=test-table [id=bf555fc6cfa042cc8300da28398a28fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:18.920375 3844 tablet_service.cc:1467] Processing CreateTablet for tablet 75b2217dedc247fab9df91297e4c23a6 (DEFAULT_TABLE table=test-table [id=bf555fc6cfa042cc8300da28398a28fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:18.920704 3582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75b2217dedc247fab9df91297e4c23a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:18.920703 3844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75b2217dedc247fab9df91297e4c23a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:18.921839 3713 tablet_service.cc:1467] Processing CreateTablet for tablet 75b2217dedc247fab9df91297e4c23a6 (DEFAULT_TABLE table=test-table [id=bf555fc6cfa042cc8300da28398a28fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:18.922137 3713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75b2217dedc247fab9df91297e4c23a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:18.923842 3925 tablet_bootstrap.cc:492] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00: Bootstrap starting.
I20260619 16:54:18.924532 3925 tablet_bootstrap.cc:654] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:18.924912 3925 log.cc:824] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:18.925006 3926 tablet_bootstrap.cc:492] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4: Bootstrap starting.
I20260619 16:54:18.925244 3924 tablet_bootstrap.cc:492] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188: Bootstrap starting.
I20260619 16:54:18.925686 3926 tablet_bootstrap.cc:654] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:18.925710 3925 tablet_bootstrap.cc:492] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00: No bootstrap required, opened a new log
I20260619 16:54:18.925763 3925 ts_tablet_manager.cc:1397] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:18.925856 3924 tablet_bootstrap.cc:654] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:18.926085 3926 log.cc:824] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:18.926119 3924 log.cc:824] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:18.926749 3926 tablet_bootstrap.cc:492] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4: No bootstrap required, opened a new log
I20260619 16:54:18.926807 3924 tablet_bootstrap.cc:492] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188: No bootstrap required, opened a new log
I20260619 16:54:18.926826 3926 ts_tablet_manager.cc:1397] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:18.926867 3924 ts_tablet_manager.cc:1397] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:18.928352 3925 raft_consensus.cc:348] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9902ef351c584921ba333472736b0188" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39757 } } peers { permanent_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35653 } } peers { permanent_uuid: "d4042405fbf64e2b8c22f337bc823b00" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 45321 } }
I20260619 16:54:18.928519 3925 raft_consensus.cc:374] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:18.928558 3925 raft_consensus.cc:729] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4042405fbf64e2b8c22f337bc823b00, State: Initialized, Role: FOLLOWER
I20260619 16:54:18.928696 3925 consensus_queue.cc:260] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9902ef351c584921ba333472736b0188" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39757 } } peers { permanent_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35653 } } peers { permanent_uuid: "d4042405fbf64e2b8c22f337bc823b00" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 45321 } }
I20260619 16:54:18.929003 3648 heartbeater.cc:503] Master 127.30.148.190:45861 was elected leader, sending a full tablet report...
I20260619 16:54:18.928951 3926 raft_consensus.cc:348] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9902ef351c584921ba333472736b0188" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39757 } } peers { permanent_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35653 } } peers { permanent_uuid: "d4042405fbf64e2b8c22f337bc823b00" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 45321 } }
I20260619 16:54:18.929136 3926 raft_consensus.cc:374] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:18.929193 3926 raft_consensus.cc:729] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89e6c93ae4c64f1cba1bdbdadf9b6ad4, State: Initialized, Role: FOLLOWER
I20260619 16:54:18.929286 3925 ts_tablet_manager.cc:1428] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260619 16:54:18.929324 3926 consensus_queue.cc:260] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9902ef351c584921ba333472736b0188" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39757 } } peers { permanent_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35653 } } peers { permanent_uuid: "d4042405fbf64e2b8c22f337bc823b00" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 45321 } }
I20260619 16:54:18.929461 3924 raft_consensus.cc:348] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9902ef351c584921ba333472736b0188" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39757 } } peers { permanent_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35653 } } peers { permanent_uuid: "d4042405fbf64e2b8c22f337bc823b00" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 45321 } }
I20260619 16:54:18.929589 3926 ts_tablet_manager.cc:1428] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:54:18.929610 3924 raft_consensus.cc:374] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:18.929646 3779 heartbeater.cc:503] Master 127.30.148.190:45861 was elected leader, sending a full tablet report...
I20260619 16:54:18.929646 3924 raft_consensus.cc:729] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9902ef351c584921ba333472736b0188, State: Initialized, Role: FOLLOWER
I20260619 16:54:18.929741 3924 consensus_queue.cc:260] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9902ef351c584921ba333472736b0188" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39757 } } peers { permanent_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35653 } } peers { permanent_uuid: "d4042405fbf64e2b8c22f337bc823b00" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 45321 } }
I20260619 16:54:18.929960 3924 ts_tablet_manager.cc:1428] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:54:18.930022 3910 heartbeater.cc:503] Master 127.30.148.190:45861 was elected leader, sending a full tablet report...
W20260619 16:54:19.034472 3780 tablet.cc:2369] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:54:19.056288 3930 raft_consensus.cc:482] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:19.056406 3930 raft_consensus.cc:504] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9902ef351c584921ba333472736b0188" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39757 } } peers { permanent_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35653 } } peers { permanent_uuid: "d4042405fbf64e2b8c22f337bc823b00" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 45321 } }
I20260619 16:54:19.056690 3930 leader_election.cc:288] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9902ef351c584921ba333472736b0188 (127.30.148.131:39757), 89e6c93ae4c64f1cba1bdbdadf9b6ad4 (127.30.148.130:35653)
I20260619 16:54:19.058117 3650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.056883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:39757 (local address 127.30.148.129:60359)
0619 16:54:19.056944 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:19.056957 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:19.057008 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:19.057201 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:19.057202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:19.057205 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:19.057718 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:19.057719 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:19.057727 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:19.057741 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:19.057944 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:19.057945 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:19.057945 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 16:54:19.058018 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 16:54:19.058033 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 16:54:19.058147 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.057014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:60359 (local address 127.30.148.131:39757)
0619 16:54:19.057062 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:19.057064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:19.057074 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:19.057102 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:19.057104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:19.057110 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:19.057196 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:19.057829 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:19.057830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:19.057831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:19.057835 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:19.057870 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:19.057875 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:19.057941 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:19.058028 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 16:54:19.058042 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:54:19.058240 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.056989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:38053 (local address 127.30.148.130:35653)
0619 16:54:19.057031 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:19.057033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:19.057213 (+ 180us) server_negotiation.cc:407] Connection header received
0619 16:54:19.057278 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:19.057279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:19.057283 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:19.057340 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:19.058018 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:19.058019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:19.058020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:19.058025 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:19.058041 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:19.058045 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:19.058101 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:19.058158 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 16:54:19.058171 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:54:19.058266 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.056884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:35653 (local address 127.30.148.129:38053)
0619 16:54:19.057141 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:19.057148 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:19.057227 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:19.057345 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:19.057345 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:19.057377 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:19.057904 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:19.057905 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:19.057912 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:19.057926 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:19.058106 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:19.058107 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:19.058108 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 16:54:19.058157 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 16:54:19.058170 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":106,"thread_start_us":97,"threads_started":1}
I20260619 16:54:19.058506 3864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b2217dedc247fab9df91297e4c23a6" candidate_uuid: "d4042405fbf64e2b8c22f337bc823b00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9902ef351c584921ba333472736b0188" is_pre_election: true
I20260619 16:54:19.058565 3733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b2217dedc247fab9df91297e4c23a6" candidate_uuid: "d4042405fbf64e2b8c22f337bc823b00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4" is_pre_election: true
I20260619 16:54:19.058657 3864 raft_consensus.cc:2454] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4042405fbf64e2b8c22f337bc823b00 in term 0.
I20260619 16:54:19.058672 3733 raft_consensus.cc:2454] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4042405fbf64e2b8c22f337bc823b00 in term 0.
I20260619 16:54:19.058867 3535 leader_election.cc:302] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9902ef351c584921ba333472736b0188, d4042405fbf64e2b8c22f337bc823b00; no voters:
I20260619 16:54:19.059036 3930 raft_consensus.cc:2781] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:19.059074 3930 raft_consensus.cc:482] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:19.059089 3930 raft_consensus.cc:3037] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:19.059722 3930 raft_consensus.cc:504] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9902ef351c584921ba333472736b0188" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39757 } } peers { permanent_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35653 } } peers { permanent_uuid: "d4042405fbf64e2b8c22f337bc823b00" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 45321 } }
I20260619 16:54:19.059855 3930 leader_election.cc:288] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [CANDIDATE]: Term 1 election: Requested vote from peers 9902ef351c584921ba333472736b0188 (127.30.148.131:39757), 89e6c93ae4c64f1cba1bdbdadf9b6ad4 (127.30.148.130:35653)
I20260619 16:54:19.060009 3733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b2217dedc247fab9df91297e4c23a6" candidate_uuid: "d4042405fbf64e2b8c22f337bc823b00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4"
I20260619 16:54:19.060009 3864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b2217dedc247fab9df91297e4c23a6" candidate_uuid: "d4042405fbf64e2b8c22f337bc823b00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9902ef351c584921ba333472736b0188"
I20260619 16:54:19.060082 3864 raft_consensus.cc:3037] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:19.060082 3733 raft_consensus.cc:3037] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:19.060937 3733 raft_consensus.cc:2454] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4042405fbf64e2b8c22f337bc823b00 in term 1.
I20260619 16:54:19.060937 3864 raft_consensus.cc:2454] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4042405fbf64e2b8c22f337bc823b00 in term 1.
I20260619 16:54:19.061106 3538 leader_election.cc:302] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89e6c93ae4c64f1cba1bdbdadf9b6ad4, d4042405fbf64e2b8c22f337bc823b00; no voters:
I20260619 16:54:19.061188 3930 raft_consensus.cc:2781] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:19.061318 3930 raft_consensus.cc:686] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [term 1 LEADER]: Becoming Leader. State: Replica: d4042405fbf64e2b8c22f337bc823b00, State: Running, Role: LEADER
I20260619 16:54:19.061440 3930 consensus_queue.cc:237] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9902ef351c584921ba333472736b0188" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39757 } } peers { permanent_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35653 } } peers { permanent_uuid: "d4042405fbf64e2b8c22f337bc823b00" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 45321 } }
I20260619 16:54:19.062214 3460 catalog_manager.cc:5495] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4042405fbf64e2b8c22f337bc823b00 (127.30.148.129). New cstate: current_term: 1 leader_uuid: "d4042405fbf64e2b8c22f337bc823b00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9902ef351c584921ba333472736b0188" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4042405fbf64e2b8c22f337bc823b00" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 45321 } health_report { overall_health: HEALTHY } } }
I20260619 16:54:19.065886 3923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.064853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49584 (local address 127.30.148.129:45321)
0619 16:54:19.064909 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:19.064911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:19.064921 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:19.065014 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:19.065015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:19.065025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:19.065074 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:19.065691 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:19.065693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:19.065694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:19.065698 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:19.065710 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:19.065713 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:19.065758 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:19.065811 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 16:54:19.065824 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 16:54:19.067337 3864 raft_consensus.cc:1261] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188 [term 1 FOLLOWER]: Refusing update from remote peer d4042405fbf64e2b8c22f337bc823b00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:19.067376 3733 raft_consensus.cc:1261] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4 [term 1 FOLLOWER]: Refusing update from remote peer d4042405fbf64e2b8c22f337bc823b00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:19.067652 3930 consensus_queue.cc:1035] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9902ef351c584921ba333472736b0188" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:19.067844 3934 consensus_queue.cc:1035] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89e6c93ae4c64f1cba1bdbdadf9b6ad4" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:19.069983 3937 mvcc.cc:205] Tried to move back new op lower bound from 7298613489937715200 to 7298613489916207104. Current Snapshot: MvccSnapshot[applied={T|T < 7298613489937715200}]
I20260619 16:54:19.071229 3939 mvcc.cc:205] Tried to move back new op lower bound from 7298613489937715200 to 7298613489916207104. Current Snapshot: MvccSnapshot[applied={T|T < 7298613489937715200 or (T in {7298613489937715200})}]
I20260619 16:54:19.071944 3460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:19.072052 3460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:19.072747 3460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0de2c4d2df2b4ee5b1cdd599d4ab8b5d: Sending DeleteTablet for 3 replicas of tablet 75b2217dedc247fab9df91297e4c23a6
I20260619 16:54:19.073046 3844 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b2217dedc247fab9df91297e4c23a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:19 UTC) from {username='slave'} at 127.0.0.1:36636
I20260619 16:54:19.073077 3713 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b2217dedc247fab9df91297e4c23a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:19 UTC) from {username='slave'} at 127.0.0.1:47624
I20260619 16:54:19.073239 3582 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b2217dedc247fab9df91297e4c23a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:19 UTC) from {username='slave'} at 127.0.0.1:49568
I20260619 16:54:19.073282 3948 tablet_replica.cc:330] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4: stopping tablet replica
I20260619 16:54:19.073401 3948 raft_consensus.cc:2229] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:19.073506 3948 raft_consensus.cc:2258] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:19.073500 3949 tablet_replica.cc:330] T 75b2217dedc247fab9df91297e4c23a6 P d4042405fbf64e2b8c22f337bc823b00: stopping tablet replica
I20260619 16:54:19.073582 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 3520
I20260619 16:54:19.077780 3947 tablet_replica.cc:330] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188: stopping tablet replica
I20260619 16:54:19.077888 3947 raft_consensus.cc:2229] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:19.077979 3947 raft_consensus.cc:2258] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:19.078399 3947 ts_tablet_manager.cc:1905] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:19.080253 3947 ts_tablet_manager.cc:1918] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:19.080338 3947 log.cc:1196] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/75b2217dedc247fab9df91297e4c23a6
I20260619 16:54:19.080585 3947 ts_tablet_manager.cc:1939] T 75b2217dedc247fab9df91297e4c23a6 P 9902ef351c584921ba333472736b0188: Deleting consensus metadata
I20260619 16:54:19.080999 3444 catalog_manager.cc:4841] TS 9902ef351c584921ba333472736b0188 (127.30.148.131:39757): tablet 75b2217dedc247fab9df91297e4c23a6 (table test-table [id=bf555fc6cfa042cc8300da28398a28fb]) successfully deleted
I20260619 16:54:19.081674 3948 ts_tablet_manager.cc:1905] T 75b2217dedc247fab9df91297e4c23a6 P 89e6c93ae4c64f1cba1bdbdadf9b6ad4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 16:54:19.082155 3445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:45321 (error 108)
I20260619 16:54:19.082425 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 3651
I20260619 16:54:19.082554 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.082430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:45321 (local address 127.0.0.1:49586)
0619 16:54:19.082482 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:19.082520 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:45321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 16:54:19.084067 3445 catalog_manager.cc:4584] TS d4042405fbf64e2b8c22f337bc823b00 (127.30.148.129:45321): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75b2217dedc247fab9df91297e4c23a6: Network error: Client connection negotiation failed: client connection to 127.30.148.129:45321: connect: Connection refused (error 111)
I20260619 16:54:19.087168 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 3782
I20260619 16:54:19.087199 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.087091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:35653 (local address 127.0.0.1:47634)
0619 16:54:19.087143 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:19.087173 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.130:35653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 16:54:19.091895 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 3429
2026-06-19T16:54:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (681 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 16:54:19.110787 31314 mini_oidc.cc:152] Starting JWKS server
I20260619 16:54:19.111763 31314 webserver.cc:460] Webserver started at https://127.0.0.1:44791/ using document root <none> and password file <none>
I20260619 16:54:19.111835 31314 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 16:54:19.111963 31314 webserver.cc:460] Webserver started at http://127.0.0.1:35503/ using document root <none> and password file <none>
2026-06-19T16:54:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:19Z Disabled control of system clock
I20260619 16:54:19.119251 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:46075
--webserver_interface=127.30.148.190
--webserver_port=0
--builtin_ntp_servers=127.30.148.148:38489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:46075
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44791/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/testchainca.cert with env {}
W20260619 16:54:19.197329 3956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:19.197587 3956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:19.197609 3956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:19.198676 3956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:19.198709 3956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:19.198721 3956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:19.198734 3956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:19.198746 3956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 16:54:19.198758 3956 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 16:54:19.198770 3956 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44791/jwks.json
I20260619 16:54:19.200192 3956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:46075
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:46075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44791/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:19.200366 3956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:19.200518 3956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:19.205070 3964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:19.205143 3956 server_base.cc:1034] running on GCE node
W20260619 16:54:19.205070 3962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:19.205057 3961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:19.205514 3956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:19.205749 3956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:19.206881 3956 hybrid_clock.cc:648] HybridClock initialized: now 1781888059206869 us; error 30 us; skew 500 ppm
I20260619 16:54:19.207986 3956 webserver.cc:460] Webserver started at http://127.30.148.190:44619/ using document root <none> and password file <none>
I20260619 16:54:19.208211 3956 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:19.208277 3956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:19.208420 3956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:19.209482 3956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "aefec5441c614fdf9f8c1f958e170b29"
format_stamp: "Formatted at 2026-06-19 16:54:19 on dist-test-slave-xb9n"
server_key: "683d75affc74409ba3d6290d1e0327c5"
server_key_iv: "48b77f51db01512508b7b092dcc6b226"
server_key_version: "encryptionkey@0"
I20260619 16:54:19.209822 3956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "aefec5441c614fdf9f8c1f958e170b29"
format_stamp: "Formatted at 2026-06-19 16:54:19 on dist-test-slave-xb9n"
server_key: "683d75affc74409ba3d6290d1e0327c5"
server_key_iv: "48b77f51db01512508b7b092dcc6b226"
server_key_version: "encryptionkey@0"
I20260619 16:54:19.211025 3956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:19.211670 3970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:19.211824 3956 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:19.211917 3956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "aefec5441c614fdf9f8c1f958e170b29"
format_stamp: "Formatted at 2026-06-19 16:54:19 on dist-test-slave-xb9n"
server_key: "683d75affc74409ba3d6290d1e0327c5"
server_key_iv: "48b77f51db01512508b7b092dcc6b226"
server_key_version: "encryptionkey@0"
I20260619 16:54:19.211989 3956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:19.226131 3956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:19.226387 3956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:19.226531 3956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:19.230171 3956 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:46075
I20260619 16:54:19.230201 4022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:46075 every 8 connection(s)
I20260619 16:54:19.230494 3956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:19.231324 4023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:19.232841 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 3956
I20260619 16:54:19.232941 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:19.233106 31314 external_mini_cluster.cc:1428] Setting key 42175f85d65e6ab189fc032734290def
I20260619 16:54:19.233741 4023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29: Bootstrap starting.
I20260619 16:54:19.234308 4023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:19.234711 4023 log.cc:824] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:19.235395 4023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29: No bootstrap required, opened a new log
I20260619 16:54:19.237728 4023 raft_consensus.cc:348] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefec5441c614fdf9f8c1f958e170b29" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 46075 } }
I20260619 16:54:19.237895 4023 raft_consensus.cc:374] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:19.237921 4023 raft_consensus.cc:729] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aefec5441c614fdf9f8c1f958e170b29, State: Initialized, Role: FOLLOWER
I20260619 16:54:19.238054 4023 consensus_queue.cc:260] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefec5441c614fdf9f8c1f958e170b29" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 46075 } }
I20260619 16:54:19.238124 4023 raft_consensus.cc:388] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:19.238145 4023 raft_consensus.cc:482] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:19.238211 4023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:19.238588 4026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.233574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33026 (local address 127.30.148.190:46075)
0619 16:54:19.233733 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:19.233739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:19.233763 (+ 24us) server_negotiation.cc:407] Connection header received
0619 16:54:19.234204 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:19.234211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:19.234265 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:19.234394 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:19.234653 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.235595 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.236664 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.237054 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.237142 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:19.237842 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:19.237845 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:19.237845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:19.238082 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:19.238139 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:19.238154 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:19.238218 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:19.238320 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 16:54:19.238406 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":58,"threads_started":1}
I20260619 16:54:19.238826 4023 raft_consensus.cc:504] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefec5441c614fdf9f8c1f958e170b29" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 46075 } }
I20260619 16:54:19.238942 4023 leader_election.cc:302] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aefec5441c614fdf9f8c1f958e170b29; no voters:
I20260619 16:54:19.239082 4023 leader_election.cc:288] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:19.239104 4028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:19.239238 4028 raft_consensus.cc:686] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [term 1 LEADER]: Becoming Leader. State: Replica: aefec5441c614fdf9f8c1f958e170b29, State: Running, Role: LEADER
I20260619 16:54:19.239307 4028 consensus_queue.cc:237] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefec5441c614fdf9f8c1f958e170b29" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 46075 } }
I20260619 16:54:19.239338 4023 sys_catalog.cc:564] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:19.239492 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:54:19.239639 4030 sys_catalog.cc:455] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aefec5441c614fdf9f8c1f958e170b29. Latest consensus state: current_term: 1 leader_uuid: "aefec5441c614fdf9f8c1f958e170b29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefec5441c614fdf9f8c1f958e170b29" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 46075 } } }
I20260619 16:54:19.239704 4030 sys_catalog.cc:458] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:19.239861 4029 sys_catalog.cc:455] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aefec5441c614fdf9f8c1f958e170b29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefec5441c614fdf9f8c1f958e170b29" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 46075 } } }
I20260619 16:54:19.239900 4029 sys_catalog.cc:458] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:19.240008 4033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:19.240450 4033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:19.248984 4033 catalog_manager.cc:1347] Generated new cluster ID: e66bddfd5bf0474b9991ee333104c7c4
I20260619 16:54:19.249050 4033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:19.261277 4033 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:19.261945 4033 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:19.271206 4033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aefec5441c614fdf9f8c1f958e170b29: Generated new TSK 0
I20260619 16:54:19.271379 4033 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 16:54:19.292141 4053 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 16:54:19.292240 4053 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 16:54:19.292446 4026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.277679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33040 (local address 127.30.148.190:46075)
0619 16:54:19.277720 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:19.277722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:19.277800 (+ 78us) server_negotiation.cc:407] Connection header received
0619 16:54:19.277898 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:19.277898 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:19.277908 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:19.277961 (+ 53us) server_negotiation.cc:226] Negotiated authn=JWT
0619 16:54:19.278110 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.278581 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.279679 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.279865 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.279916 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:19.280136 (+ 220us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 16:54:19.292248 (+ 12112us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 16:54:19.292402 (+ 154us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33040: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 16:54:19.292531 4026 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33040: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 16:54:19.292838 4052 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:19.277555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:46075 (local address 127.0.0.1:33040)
0619 16:54:19.277730 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:19.277741 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:19.277812 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:19.277968 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:19.277969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:19.277970 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0619 16:54:19.278046 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:19.278048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.278661 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.278662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:19.279600 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:19.279603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.279939 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.279941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:19.280014 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:19.280035 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 16:54:19.292419 (+ 12384us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 16:54:19.292536 (+ 117us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.148.190:46075: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":96,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.148.190:46075: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 16:54:19.293316 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 3956
2026-06-19T16:54:19Z chronyd exiting
I20260619 16:54:19.328953 31314 test_util.cc:175] -----------------------------------------------
I20260619 16:54:19.329041 31314 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888036930978-31314-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 16:54:19.329864 31314 mini_oidc.cc:152] Starting JWKS server
I20260619 16:54:19.330521 31314 webserver.cc:460] Webserver started at https://127.0.0.1:43865/ using document root <none> and password file <none>
I20260619 16:54:19.330586 31314 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 16:54:19.330746 31314 webserver.cc:460] Webserver started at http://127.0.0.1:36905/ using document root <none> and password file <none>
2026-06-19T16:54:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:19Z Disabled control of system clock
I20260619 16:54:19.338235 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:43003
--webserver_interface=127.30.148.190
--webserver_port=0
--builtin_ntp_servers=127.30.148.148:44543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:43003
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43865/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/testchainca.cert with env {}
W20260619 16:54:19.416740 4059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:19.416939 4059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:19.416961 4059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:19.418043 4059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:19.418079 4059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:19.418092 4059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:19.418104 4059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:19.418116 4059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 16:54:19.418128 4059 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 16:54:19.418140 4059 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43865/jwks.json
I20260619 16:54:19.419528 4059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:43003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:43003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43865/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:19.419709 4059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:19.419862 4059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:19.424489 4065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:19.424680 4059 server_base.cc:1034] running on GCE node
W20260619 16:54:19.424506 4067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:19.424489 4064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:19.425004 4059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:19.425253 4059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:19.426422 4059 hybrid_clock.cc:648] HybridClock initialized: now 1781888059426402 us; error 32 us; skew 500 ppm
I20260619 16:54:19.427714 4059 webserver.cc:460] Webserver started at http://127.30.148.190:42289/ using document root <none> and password file <none>
I20260619 16:54:19.427948 4059 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:19.428018 4059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:19.428166 4059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:19.429168 4059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "ca75068a19754968805095ab5d67cb17"
format_stamp: "Formatted at 2026-06-19 16:54:19 on dist-test-slave-xb9n"
server_key: "4d2771d2601642ba5f5417574757bf0f"
server_key_iv: "361a0e2019689e7cab234233042d9a22"
server_key_version: "encryptionkey@0"
I20260619 16:54:19.429559 4059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "ca75068a19754968805095ab5d67cb17"
format_stamp: "Formatted at 2026-06-19 16:54:19 on dist-test-slave-xb9n"
server_key: "4d2771d2601642ba5f5417574757bf0f"
server_key_iv: "361a0e2019689e7cab234233042d9a22"
server_key_version: "encryptionkey@0"
I20260619 16:54:19.430863 4059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:19.431640 4073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:19.431855 4059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:19.431964 4059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "ca75068a19754968805095ab5d67cb17"
format_stamp: "Formatted at 2026-06-19 16:54:19 on dist-test-slave-xb9n"
server_key: "4d2771d2601642ba5f5417574757bf0f"
server_key_iv: "361a0e2019689e7cab234233042d9a22"
server_key_version: "encryptionkey@0"
I20260619 16:54:19.432045 4059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:19.439136 4059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:19.439428 4059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:19.439570 4059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:19.443559 4059 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:43003
I20260619 16:54:19.443591 4125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:43003 every 8 connection(s)
I20260619 16:54:19.443931 4059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:19.444541 4126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:19.447078 4126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17: Bootstrap starting.
I20260619 16:54:19.447644 4126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:19.447914 4126 log.cc:824] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:19.448587 4126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17: No bootstrap required, opened a new log
I20260619 16:54:19.450239 4126 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca75068a19754968805095ab5d67cb17" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43003 } }
I20260619 16:54:19.450371 4126 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:19.450394 4126 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca75068a19754968805095ab5d67cb17, State: Initialized, Role: FOLLOWER
I20260619 16:54:19.450555 4126 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca75068a19754968805095ab5d67cb17" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43003 } }
I20260619 16:54:19.450625 4126 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:19.450666 4126 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:19.450716 4126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:19.451227 4126 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca75068a19754968805095ab5d67cb17" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43003 } }
I20260619 16:54:19.451337 4126 leader_election.cc:302] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca75068a19754968805095ab5d67cb17; no voters:
I20260619 16:54:19.451514 4126 leader_election.cc:288] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:19.451575 4129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:19.451771 4129 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [term 1 LEADER]: Becoming Leader. State: Replica: ca75068a19754968805095ab5d67cb17, State: Running, Role: LEADER
I20260619 16:54:19.451756 4126 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:19.451910 4129 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca75068a19754968805095ab5d67cb17" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43003 } }
I20260619 16:54:19.452236 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 4059
I20260619 16:54:19.452344 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:19.452337 4130 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca75068a19754968805095ab5d67cb17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca75068a19754968805095ab5d67cb17" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43003 } } }
I20260619 16:54:19.452425 4130 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:19.452572 31314 external_mini_cluster.cc:1428] Setting key 670d5bf84a3c6890757e3d7d6d7d9525
I20260619 16:54:19.452620 4138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:19.453003 4131 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca75068a19754968805095ab5d67cb17. Latest consensus state: current_term: 1 leader_uuid: "ca75068a19754968805095ab5d67cb17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca75068a19754968805095ab5d67cb17" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43003 } } }
I20260619 16:54:19.453137 4131 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:19.453137 4138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:19.455226 4138 catalog_manager.cc:1347] Generated new cluster ID: d27be350f2ea4d4ea77ac2db5484a79c
I20260619 16:54:19.455276 4138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:19.458379 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.453206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55048 (local address 127.30.148.190:43003)
0619 16:54:19.453400 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:19.453410 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:19.453640 (+ 230us) server_negotiation.cc:407] Connection header received
0619 16:54:19.453791 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:19.453798 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:19.453870 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:19.454004 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:19.454307 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.455475 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.456416 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.456862 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.456992 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:19.457732 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:19.457736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:19.457736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:19.457909 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:19.457944 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:19.457954 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:19.458028 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:19.458101 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 16:54:19.458182 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":69,"threads_started":1}
I20260619 16:54:19.459227 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:54:19.490998 4138 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:19.491727 4138 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:19.504932 4138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca75068a19754968805095ab5d67cb17: Generated new TSK 0
I20260619 16:54:19.505133 4138 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 16:54:19.535271 4156 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 16:54:19.535351 4156 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 16:54:19.535633 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.514398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55060 (local address 127.30.148.190:43003)
0619 16:54:19.514458 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:19.514460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:19.514546 (+ 86us) server_negotiation.cc:407] Connection header received
0619 16:54:19.514611 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:19.514612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:19.514622 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:19.514680 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0619 16:54:19.514820 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.515287 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.516418 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.516598 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.516643 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:19.516831 (+ 188us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 16:54:19.535385 (+ 18554us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 16:54:19.535538 (+ 153us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55060: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":40}
W20260619 16:54:19.535696 4147 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55060: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 16:54:19.535835 4155 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:19.514306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:43003 (local address 127.0.0.1:55060)
0619 16:54:19.514475 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:19.514486 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:19.514551 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:19.514695 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:19.514696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:19.514698 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 16:54:19.514760 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:19.514762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.515374 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.515376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:19.516333 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:19.516336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.516673 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.516674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:19.516737 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:19.516743 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 16:54:19.535589 (+ 18846us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 16:54:19.535700 (+ 111us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.148.190:43003: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
I20260619 16:54:19.536139 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 4059
2026-06-19T16:54:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (243 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 16:54:19.572906 31314 mini_oidc.cc:152] Starting JWKS server
I20260619 16:54:19.573616 31314 webserver.cc:460] Webserver started at https://127.0.0.1:34955/ using document root <none> and password file <none>
I20260619 16:54:19.573657 31314 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 16:54:19.573769 31314 webserver.cc:460] Webserver started at http://127.0.0.1:46267/ using document root <none> and password file <none>
2026-06-19T16:54:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:19Z Disabled control of system clock
I20260619 16:54:19.581101 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:46471
--webserver_interface=127.30.148.190
--webserver_port=0
--builtin_ntp_servers=127.30.148.148:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:46471
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34955/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 16:54:19.659390 4162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:19.659606 4162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:19.659641 4162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:19.660769 4162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:19.660833 4162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:19.660856 4162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:19.660876 4162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:19.660907 4162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 16:54:19.660936 4162 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 16:54:19.660960 4162 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34955/jwks.json
I20260619 16:54:19.662320 4162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:46471
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:46471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34955/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:19.662582 4162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:19.662864 4162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:19.667554 4167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:19.667719 4162 server_base.cc:1034] running on GCE node
W20260619 16:54:19.667573 4168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:19.667573 4170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:19.668016 4162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:19.668298 4162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:19.669426 4162 hybrid_clock.cc:648] HybridClock initialized: now 1781888059669416 us; error 38 us; skew 500 ppm
I20260619 16:54:19.670609 4162 webserver.cc:460] Webserver started at http://127.30.148.190:42293/ using document root <none> and password file <none>
I20260619 16:54:19.670857 4162 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:19.670923 4162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:19.671072 4162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:19.672051 4162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "d1935940e09d4164893387a64403e305"
format_stamp: "Formatted at 2026-06-19 16:54:19 on dist-test-slave-xb9n"
server_key: "67759d0738cf32bb0694603494146078"
server_key_iv: "2e0856933f2bbf05a84144848da66edd"
server_key_version: "encryptionkey@0"
I20260619 16:54:19.672386 4162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "d1935940e09d4164893387a64403e305"
format_stamp: "Formatted at 2026-06-19 16:54:19 on dist-test-slave-xb9n"
server_key: "67759d0738cf32bb0694603494146078"
server_key_iv: "2e0856933f2bbf05a84144848da66edd"
server_key_version: "encryptionkey@0"
I20260619 16:54:19.673767 4162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:19.674551 4176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:19.674741 4162 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:19.674840 4162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "d1935940e09d4164893387a64403e305"
format_stamp: "Formatted at 2026-06-19 16:54:19 on dist-test-slave-xb9n"
server_key: "67759d0738cf32bb0694603494146078"
server_key_iv: "2e0856933f2bbf05a84144848da66edd"
server_key_version: "encryptionkey@0"
I20260619 16:54:19.674919 4162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:19.684077 4162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:19.684335 4162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:19.684476 4162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:19.688179 4162 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:46471
I20260619 16:54:19.688236 4228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:46471 every 8 connection(s)
I20260619 16:54:19.688524 4162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:19.689136 4229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:19.691646 4229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305: Bootstrap starting.
I20260619 16:54:19.692199 4229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:19.692493 4229 log.cc:824] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:19.693140 4229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305: No bootstrap required, opened a new log
I20260619 16:54:19.694777 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 4162
I20260619 16:54:19.694866 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:19.694749 4229 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1935940e09d4164893387a64403e305" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 46471 } }
I20260619 16:54:19.694924 4229 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:19.694975 4229 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1935940e09d4164893387a64403e305, State: Initialized, Role: FOLLOWER
I20260619 16:54:19.695080 31314 external_mini_cluster.cc:1428] Setting key 4d5fb72d12e518912cbe4a1ebe3e4a52
I20260619 16:54:19.695137 4229 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1935940e09d4164893387a64403e305" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 46471 } }
I20260619 16:54:19.695226 4229 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:19.695268 4229 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:19.695312 4229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:19.695990 4229 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1935940e09d4164893387a64403e305" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 46471 } }
I20260619 16:54:19.696076 4229 leader_election.cc:302] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1935940e09d4164893387a64403e305; no voters:
I20260619 16:54:19.696257 4229 leader_election.cc:288] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:19.696285 4234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:19.696386 4234 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [term 1 LEADER]: Becoming Leader. State: Replica: d1935940e09d4164893387a64403e305, State: Running, Role: LEADER
I20260619 16:54:19.696465 4234 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1935940e09d4164893387a64403e305" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 46471 } }
I20260619 16:54:19.696496 4229 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:19.696807 4234 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1935940e09d4164893387a64403e305. Latest consensus state: current_term: 1 leader_uuid: "d1935940e09d4164893387a64403e305" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1935940e09d4164893387a64403e305" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 46471 } } }
I20260619 16:54:19.696883 4234 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:19.697185 4236 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1935940e09d4164893387a64403e305" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1935940e09d4164893387a64403e305" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 46471 } } }
I20260619 16:54:19.697270 4236 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:19.697314 4240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:19.697847 4240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:19.699532 4240 catalog_manager.cc:1347] Generated new cluster ID: 25f982c3d232431f9aa3f6ca0fdfa93b
I20260619 16:54:19.699582 4240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:19.701927 4233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.695621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36138 (local address 127.30.148.190:46471)
0619 16:54:19.695806 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:19.695815 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:19.695847 (+ 32us) server_negotiation.cc:407] Connection header received
0619 16:54:19.695950 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:19.695956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:19.696011 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:19.696148 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:19.696510 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.698933 (+ 2423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.699884 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.700453 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.700584 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:19.701208 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:19.701212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:19.701212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:19.701461 (+ 249us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:19.701510 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:19.701524 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:19.701593 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:19.701657 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 16:54:19.701729 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260619 16:54:19.702811 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:54:19.715092 4240 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:19.715552 4240 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:19.718061 4240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1935940e09d4164893387a64403e305: Generated new TSK 0
I20260619 16:54:19.718214 4240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:54:19.726681 4233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.723150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.30.148.190:46471)
0619 16:54:19.723194 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:19.723196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:19.723299 (+ 103us) server_negotiation.cc:407] Connection header received
0619 16:54:19.723336 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:19.723337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:19.723348 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:19.723422 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0619 16:54:19.723542 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.724311 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.725026 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.725381 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.725448 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:19.725587 (+ 139us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 16:54:19.726469 (+ 882us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34955/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 16:54:19.726611 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36148: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34955/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 16:54:19.726732 4233 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36148: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34955/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 16:54:19.728615 4258 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:19.723050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:46471 (local address 127.0.0.1:36148)
0619 16:54:19.723235 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:19.723244 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:19.723294 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:19.723428 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:19.723428 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:19.723430 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 16:54:19.723477 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:19.723479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.724391 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.724391 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:19.724947 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:19.724948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.725459 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.725460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:19.725501 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:19.725504 (+ 3us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 16:54:19.726579 (+ 1075us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34955/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 16:54:19.728518 (+ 1939us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.148.190:46471: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34955/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":1756,"thread_start_us":102,"threads_started":1}
I20260619 16:54:19.728809 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 4162
E20260619 16:54:19.741380 4259 webserver.cc:533] Webserver: sslize failed:
E20260619 16:54:19.741478 4259 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T16:54:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (192 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T16:54:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:19Z Disabled control of system clock
I20260619 16:54:19.772915 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:34481
--webserver_interface=127.30.148.190
--webserver_port=0
--builtin_ntp_servers=127.30.148.148:39919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:34481
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/testchainca.cert with env {}
W20260619 16:54:19.851343 4263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:19.851521 4263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:19.851541 4263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:19.852603 4263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:19.852638 4263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:19.852649 4263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:19.852662 4263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:19.852674 4263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 16:54:19.852686 4263 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 16:54:19.852697 4263 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 16:54:19.854113 4263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:34481
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:34481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:19.854296 4263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:19.854451 4263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:19.858940 4268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:19.858940 4269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:19.859011 4271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:19.859072 4263 server_base.cc:1034] running on GCE node
I20260619 16:54:19.859349 4263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:19.859620 4263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:19.860762 4263 hybrid_clock.cc:648] HybridClock initialized: now 1781888059860742 us; error 38 us; skew 500 ppm
I20260619 16:54:19.861941 4263 webserver.cc:460] Webserver started at http://127.30.148.190:38429/ using document root <none> and password file <none>
I20260619 16:54:19.862185 4263 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:19.862236 4263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:19.862402 4263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:19.863399 4263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "986ebbffaf0546beb5eb6d05e71bad02"
format_stamp: "Formatted at 2026-06-19 16:54:19 on dist-test-slave-xb9n"
server_key: "ce341408dc47fa609e9b307d39d3886d"
server_key_iv: "4f4f43e1da7e75b529d5db73ecbc3ac9"
server_key_version: "encryptionkey@0"
I20260619 16:54:19.863742 4263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "986ebbffaf0546beb5eb6d05e71bad02"
format_stamp: "Formatted at 2026-06-19 16:54:19 on dist-test-slave-xb9n"
server_key: "ce341408dc47fa609e9b307d39d3886d"
server_key_iv: "4f4f43e1da7e75b529d5db73ecbc3ac9"
server_key_version: "encryptionkey@0"
I20260619 16:54:19.864957 4263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 16:54:19.865743 4277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:19.865921 4263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:19.866006 4263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "986ebbffaf0546beb5eb6d05e71bad02"
format_stamp: "Formatted at 2026-06-19 16:54:19 on dist-test-slave-xb9n"
server_key: "ce341408dc47fa609e9b307d39d3886d"
server_key_iv: "4f4f43e1da7e75b529d5db73ecbc3ac9"
server_key_version: "encryptionkey@0"
I20260619 16:54:19.866077 4263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:19.878611 4263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:19.878888 4263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:19.879033 4263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:19.883023 4263 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:34481
I20260619 16:54:19.883056 4329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:34481 every 8 connection(s)
I20260619 16:54:19.883447 4263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:19.884318 4330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:19.886394 4330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02: Bootstrap starting.
I20260619 16:54:19.886725 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 4263
I20260619 16:54:19.886826 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:19.886979 4330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:19.887025 31314 external_mini_cluster.cc:1428] Setting key e41e3e22f66dd04ab4b11a5713f9a247
I20260619 16:54:19.887262 4330 log.cc:824] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:19.888002 4330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02: No bootstrap required, opened a new log
I20260619 16:54:19.890141 4330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986ebbffaf0546beb5eb6d05e71bad02" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 34481 } }
I20260619 16:54:19.890298 4330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:19.890328 4330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 986ebbffaf0546beb5eb6d05e71bad02, State: Initialized, Role: FOLLOWER
I20260619 16:54:19.890432 4330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986ebbffaf0546beb5eb6d05e71bad02" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 34481 } }
I20260619 16:54:19.890512 4330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:19.890543 4330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:19.890589 4330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:19.891144 4330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986ebbffaf0546beb5eb6d05e71bad02" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 34481 } }
I20260619 16:54:19.891251 4330 leader_election.cc:302] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 986ebbffaf0546beb5eb6d05e71bad02; no voters:
I20260619 16:54:19.891408 4330 leader_election.cc:288] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:19.891605 4336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:19.891702 4330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:19.891752 4336 raft_consensus.cc:686] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [term 1 LEADER]: Becoming Leader. State: Replica: 986ebbffaf0546beb5eb6d05e71bad02, State: Running, Role: LEADER
I20260619 16:54:19.891842 4336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986ebbffaf0546beb5eb6d05e71bad02" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 34481 } }
I20260619 16:54:19.892138 4336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 986ebbffaf0546beb5eb6d05e71bad02. Latest consensus state: current_term: 1 leader_uuid: "986ebbffaf0546beb5eb6d05e71bad02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986ebbffaf0546beb5eb6d05e71bad02" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 34481 } } }
I20260619 16:54:19.892199 4336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:19.892319 4335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "986ebbffaf0546beb5eb6d05e71bad02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986ebbffaf0546beb5eb6d05e71bad02" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 34481 } } }
I20260619 16:54:19.892441 4335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:19.892753 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.887464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.30.148.190:34481)
0619 16:54:19.887689 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:19.887698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:19.887748 (+ 50us) server_negotiation.cc:407] Connection header received
0619 16:54:19.888226 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:19.888233 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:19.888290 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:19.888402 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:19.888651 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.889548 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.890663 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.891066 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.891153 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:19.891985 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:19.891989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:19.891989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:19.892210 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:19.892250 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:19.892262 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:19.892312 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:19.892473 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 16:54:19.892558 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":83,"threads_started":1}
W20260619 16:54:19.893563 4351 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:54:19.893643 4351 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:54:19.893695 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:54:19.894258 4341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:19.894384 4341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:19.895689 4341 catalog_manager.cc:1347] Generated new cluster ID: 78eb952371ba4370a5f79184555f8f02
I20260619 16:54:19.895745 4341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:19.931643 4341 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:19.932387 4341 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:19.935632 4341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 986ebbffaf0546beb5eb6d05e71bad02: Generated new TSK 0
I20260619 16:54:19.935796 4341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:54:19.992101 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:19.945241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49656 (local address 127.30.148.190:34481)
0619 16:54:19.945297 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:19.945299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:19.945425 (+ 126us) server_negotiation.cc:407] Connection header received
0619 16:54:19.945516 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:19.945517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:19.945529 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:19.945598 (+ 69us) server_negotiation.cc:226] Negotiated authn=JWT
0619 16:54:19.945709 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.946417 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.947141 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.947436 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.947518 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:19.947666 (+ 148us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 16:54:19.991846 (+ 44180us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 16:54:19.992030 (+ 184us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49656: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 16:54:19.992175 4334 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49656: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 16:54:19.992197 4359 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:19.945188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:34481 (local address 127.0.0.1:49656)
0619 16:54:19.945347 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:19.945378 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:19.945437 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:19.945601 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:19.945602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:19.945603 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0619 16:54:19.945652 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:19.945654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.946492 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.946492 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:19.947067 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:19.947069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:19.947534 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:19.947534 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:19.947579 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:19.947583 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 16:54:19.991979 (+ 44396us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 16:54:19.992095 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.148.190:34481: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":107,"threads_started":1}
I20260619 16:54:19.992656 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 4263
2026-06-19T16:54:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (243 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (87 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (85 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:20 dist-test-slave-xb9n krb5kdc[4371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:20 dist-test-slave-xb9n krb5kdc[4371](info): set up 2 sockets
Jun 19 16:54:20 dist-test-slave-xb9n krb5kdc[4371](info): commencing operation
krb5kdc: starting...
W20260619 16:54:21.185539 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:54:21.295750 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:41071
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:42345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:41071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:21.373019 4384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:21.373251 4384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:21.373289 4384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:21.374454 4384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:21.374523 4384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:21.374547 4384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:21.374567 4384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:21.374595 4384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:21.376035 4384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:42345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:41071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:41071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:21.376297 4384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:21.376523 4384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:54:21.381253 4384 server_base.cc:1034] running on GCE node
W20260619 16:54:21.381259 4389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:21.381256 4390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:21.381399 4392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:21.381736 4384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:21.381976 4384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:21.383114 4384 hybrid_clock.cc:648] HybridClock initialized: now 1781888061383091 us; error 40 us; skew 500 ppm
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:21.385553 4384 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:54:21.385934 4384 webserver.cc:460] Webserver started at http://127.30.148.190:35147/ using document root <none> and password file <none>
I20260619 16:54:21.386111 4384 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:21.386150 4384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:21.386245 4384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:21.387208 4384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "cd64739a469746cba233c874a449e878"
format_stamp: "Formatted at 2026-06-19 16:54:21 on dist-test-slave-xb9n"
server_key: "bf3a29db0b1aabc8d8e00cc713b615a1"
server_key_iv: "8ab0a4ac4d901ec1b9dd6ddd7bbb61f7"
server_key_version: "encryptionkey@0"
I20260619 16:54:21.387526 4384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "cd64739a469746cba233c874a449e878"
format_stamp: "Formatted at 2026-06-19 16:54:21 on dist-test-slave-xb9n"
server_key: "bf3a29db0b1aabc8d8e00cc713b615a1"
server_key_iv: "8ab0a4ac4d901ec1b9dd6ddd7bbb61f7"
server_key_version: "encryptionkey@0"
I20260619 16:54:21.388777 4384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 16:54:21.389695 4399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:21.389861 4384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:21.389969 4384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "cd64739a469746cba233c874a449e878"
format_stamp: "Formatted at 2026-06-19 16:54:21 on dist-test-slave-xb9n"
server_key: "bf3a29db0b1aabc8d8e00cc713b615a1"
server_key_iv: "8ab0a4ac4d901ec1b9dd6ddd7bbb61f7"
server_key_version: "encryptionkey@0"
I20260619 16:54:21.390053 4384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:21.400538 4384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:21.402288 4384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:21.402446 4384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:21.405994 4384 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:41071
I20260619 16:54:21.406028 4451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:41071 every 8 connection(s)
I20260619 16:54:21.406373 4384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:21.407171 4452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:21.409210 4452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878: Bootstrap starting.
I20260619 16:54:21.409629 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 4384
I20260619 16:54:21.409706 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:21.409816 4452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:21.409878 31314 external_mini_cluster.cc:1428] Setting key 951003f1213081e2f2ca26ed399c3f8b
I20260619 16:54:21.410152 4452 log.cc:824] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:21.410919 4452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878: No bootstrap required, opened a new log
I20260619 16:54:21.413210 4452 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd64739a469746cba233c874a449e878" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41071 } }
I20260619 16:54:21.413408 4452 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:21.413455 4452 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd64739a469746cba233c874a449e878, State: Initialized, Role: FOLLOWER
I20260619 16:54:21.413566 4452 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd64739a469746cba233c874a449e878" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41071 } }
I20260619 16:54:21.413633 4452 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:21.413659 4452 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:21.413693 4452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:21.414299 4452 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd64739a469746cba233c874a449e878" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41071 } }
I20260619 16:54:21.414384 4452 leader_election.cc:302] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd64739a469746cba233c874a449e878; no voters:
I20260619 16:54:21.414588 4452 leader_election.cc:288] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:21.414613 4457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:21.414726 4457 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [term 1 LEADER]: Becoming Leader. State: Replica: cd64739a469746cba233c874a449e878, State: Running, Role: LEADER
I20260619 16:54:21.414799 4457 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd64739a469746cba233c874a449e878" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41071 } }
I20260619 16:54:21.414839 4452 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:21.415083 4457 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd64739a469746cba233c874a449e878. Latest consensus state: current_term: 1 leader_uuid: "cd64739a469746cba233c874a449e878" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd64739a469746cba233c874a449e878" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41071 } } }
I20260619 16:54:21.415151 4457 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:21.415294 4458 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd64739a469746cba233c874a449e878" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd64739a469746cba233c874a449e878" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41071 } } }
I20260619 16:54:21.415372 4458 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878 [sys.catalog]: This master's current role is: LEADER
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
W20260619 16:54:21.416607 4472 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:54:21.417284 4472 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:54:21.417382 4462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:21.417554 4462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:21.419214 4462 catalog_manager.cc:1347] Generated new cluster ID: 3f0c53382c00457693d15096f94aca74
I20260619 16:54:21.419261 4462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:21.420696 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:21.410360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38010 (local address 127.30.148.190:41071)
0619 16:54:21.410523 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:21.410536 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:21.410731 (+ 195us) server_negotiation.cc:407] Connection header received
0619 16:54:21.411212 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:21.411219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:21.411261 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:21.411367 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:21.412121 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.412915 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.413985 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.414392 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.414495 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:21.417138 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:21.417141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:21.417142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:21.417154 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:21.419069 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.419469 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.419472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.419473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.419580 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.419790 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.419791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.419791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.420099 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:21.420264 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:21.420341 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 16:54:21.420473 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260619 16:54:21.435564 4462 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:21.436069 4462 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:21.441911 4462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd64739a469746cba233c874a449e878: Generated new TSK 0
I20260619 16:54:21.442088 4462 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:54:21.466425 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41071
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:42345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:21.545004 4478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:21.545186 4478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:21.545209 4478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:21.546342 4478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:21.546383 4478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:21.546408 4478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:21.547868 4478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:42345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:21.548096 4478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:21.548324 4478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:21.553172 4486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:21.553186 4484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:21.553208 4478 server_base.cc:1034] running on GCE node
W20260619 16:54:21.553175 4483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:21.553602 4478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:21.553853 4478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:21.555013 4478 hybrid_clock.cc:648] HybridClock initialized: now 1781888061554992 us; error 34 us; skew 500 ppm
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:21.557484 4478 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:54:21.557994 4478 webserver.cc:460] Webserver started at http://127.30.148.129:45717/ using document root <none> and password file <none>
I20260619 16:54:21.558277 4478 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:21.558348 4478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:21.558499 4478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:21.559580 4478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "e8ee3092db774f73a406a74d29ee5c36"
format_stamp: "Formatted at 2026-06-19 16:54:21 on dist-test-slave-xb9n"
server_key: "c58aef007ada9d66dbabdfbc4085b7fc"
server_key_iv: "3d83ee1a793529832cc2347d6538e41b"
server_key_version: "encryptionkey@0"
I20260619 16:54:21.559929 4478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "e8ee3092db774f73a406a74d29ee5c36"
format_stamp: "Formatted at 2026-06-19 16:54:21 on dist-test-slave-xb9n"
server_key: "c58aef007ada9d66dbabdfbc4085b7fc"
server_key_iv: "3d83ee1a793529832cc2347d6538e41b"
server_key_version: "encryptionkey@0"
I20260619 16:54:21.561169 4478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:21.561967 4493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:21.562129 4478 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:21.562242 4478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "e8ee3092db774f73a406a74d29ee5c36"
format_stamp: "Formatted at 2026-06-19 16:54:21 on dist-test-slave-xb9n"
server_key: "c58aef007ada9d66dbabdfbc4085b7fc"
server_key_iv: "3d83ee1a793529832cc2347d6538e41b"
server_key_version: "encryptionkey@0"
I20260619 16:54:21.562326 4478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:21.588378 4478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:21.590222 4478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:21.590377 4478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:21.590597 4478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:21.590902 4478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:21.590961 4478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:21.590996 4478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:21.591022 4478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:21.596937 4478 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:34667
I20260619 16:54:21.596984 4606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:34667 every 8 connection(s)
I20260619 16:54:21.597272 4478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:21.601195 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 4478
I20260619 16:54:21.601284 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:21.601480 31314 external_mini_cluster.cc:1428] Setting key efa0c52a50f0b74cf181f5966aaf9dd6
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:21.606985 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:21.597968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:46347 (local address 127.30.148.190:41071)
0619 16:54:21.598035 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:21.598037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:21.598257 (+ 220us) server_negotiation.cc:407] Connection header received
0619 16:54:21.599145 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:21.599146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:21.599155 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:21.599209 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:21.599770 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.600252 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.601177 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.601509 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.601592 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:21.604199 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:21.604201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:21.604201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:21.604207 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:21.605566 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.606057 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.606058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.606058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.606108 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.606381 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.606382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.606382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.606476 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:21.606525 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:21.606862 (+ 337us) server_negotiation.cc:299] Negotiation successful
0619 16:54:21.606923 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 16:54:21.607306 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:21.597946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41071 (local address 127.30.148.129:46347)
0619 16:54:21.598127 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:21.598166 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:21.599057 (+ 891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:21.599286 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:21.599290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:21.599515 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:21.599689 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:21.599692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.600327 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.600327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:21.601091 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:21.601093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.601656 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.601657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:21.601827 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:21.602338 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:21.602349 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:21.604113 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:21.605645 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:21.605653 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:21.605655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:21.605981 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:21.606176 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:21.606177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:21.606178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:21.606311 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:21.606544 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:21.606550 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:21.606720 (+ 170us) client_negotiation.cc:769] Sending connection context
0619 16:54:21.606846 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 16:54:21.607021 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
I20260619 16:54:21.607879 4607 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41071
I20260619 16:54:21.607971 4607 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:21.608165 4607 heartbeater.cc:511] Master 127.30.148.190:41071 requested a full tablet report, sending...
I20260619 16:54:21.608618 4416 ts_manager.cc:194] Registered new tserver with Master: e8ee3092db774f73a406a74d29ee5c36 (127.30.148.129:34667)
I20260619 16:54:21.609200 4416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:46347
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:54:21.625496 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41071
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:42345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:21.703100 4612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:21.703281 4612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:21.703305 4612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:21.704413 4612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:21.704454 4612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:21.704505 4612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:21.705996 4612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:42345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:21.706218 4612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:21.706429 4612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:21.711174 4617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:21.711261 4618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:21.711182 4620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:21.711355 4612 server_base.cc:1034] running on GCE node
I20260619 16:54:21.711576 4612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:21.711854 4612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:21.713019 4612 hybrid_clock.cc:648] HybridClock initialized: now 1781888061713008 us; error 25 us; skew 500 ppm
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:21.715632 4612 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:54:21.715977 4612 webserver.cc:460] Webserver started at http://127.30.148.130:37431/ using document root <none> and password file <none>
I20260619 16:54:21.716143 4612 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:21.716181 4612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:21.716270 4612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:21.717232 4612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "ee74d2d882ba46638c471364b5d01176"
format_stamp: "Formatted at 2026-06-19 16:54:21 on dist-test-slave-xb9n"
server_key: "eb715a86979c82d9354eb5d7c4a1de6f"
server_key_iv: "5ec7e09ac23906c1c563d53a7f9ea9f7"
server_key_version: "encryptionkey@0"
I20260619 16:54:21.717583 4612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "ee74d2d882ba46638c471364b5d01176"
format_stamp: "Formatted at 2026-06-19 16:54:21 on dist-test-slave-xb9n"
server_key: "eb715a86979c82d9354eb5d7c4a1de6f"
server_key_iv: "5ec7e09ac23906c1c563d53a7f9ea9f7"
server_key_version: "encryptionkey@0"
I20260619 16:54:21.718747 4612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:21.719383 4627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:21.719532 4612 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:21.719614 4612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "ee74d2d882ba46638c471364b5d01176"
format_stamp: "Formatted at 2026-06-19 16:54:21 on dist-test-slave-xb9n"
server_key: "eb715a86979c82d9354eb5d7c4a1de6f"
server_key_iv: "5ec7e09ac23906c1c563d53a7f9ea9f7"
server_key_version: "encryptionkey@0"
I20260619 16:54:21.719666 4612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:21.734251 4612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:21.735896 4612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:21.736001 4612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:21.736171 4612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:21.736397 4612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:21.736426 4612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:21.736446 4612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:21.736459 4612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:21.741919 4612 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:43875
I20260619 16:54:21.741998 4740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:43875 every 8 connection(s)
I20260619 16:54:21.742281 4612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:21.749557 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 4612
I20260619 16:54:21.749673 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:21.749879 31314 external_mini_cluster.cc:1428] Setting key c15b70acbdb6a8f31f649ffdee8bf445
I20260619 16:54:21.752501 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:21.743020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:43009 (local address 127.30.148.190:41071)
0619 16:54:21.743103 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:21.743105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:21.743324 (+ 219us) server_negotiation.cc:407] Connection header received
0619 16:54:21.744181 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:21.744183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:21.744193 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:21.744255 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:21.744818 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.745284 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.746267 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.746445 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.746517 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:21.748521 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:21.748523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:21.748523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:21.748532 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:21.750855 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.751357 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.751358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.751358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.751443 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.751815 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.751816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.751817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.751942 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:21.752043 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:21.752332 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 16:54:21.752411 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 16:54:21.752702 4743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:21.742964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41071 (local address 127.30.148.130:43009)
0619 16:54:21.743174 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:21.743211 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:21.744072 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:21.744322 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:21.744325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:21.744572 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:21.744745 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:21.744748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.745398 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.745399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:21.746193 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:21.746195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.746521 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.746522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:21.746626 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:21.747118 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:21.747126 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:21.748410 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:21.750943 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:21.750949 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:21.750951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:21.751266 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:21.751511 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:21.751512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:21.751512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:21.751735 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:21.752046 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:21.752053 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:21.752189 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 16:54:21.752311 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 16:54:21.752462 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":142,"threads_started":1}
I20260619 16:54:21.753288 4741 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41071
I20260619 16:54:21.753407 4741 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:21.753562 4741 heartbeater.cc:511] Master 127.30.148.190:41071 requested a full tablet report, sending...
I20260619 16:54:21.753882 4416 ts_manager.cc:194] Registered new tserver with Master: ee74d2d882ba46638c471364b5d01176 (127.30.148.130:43875)
I20260619 16:54:21.754264 4416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:43009
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:54:21.773574 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41071
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:42345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:21.855897 4746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:21.856092 4746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:21.856115 4746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:21.857256 4746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:21.857293 4746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:21.857318 4746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:21.858774 4746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:42345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:21.858954 4746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:21.859118 4746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:21.864123 4751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:21.864174 4754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:21.864130 4752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:21.864161 4746 server_base.cc:1034] running on GCE node
I20260619 16:54:21.864694 4746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:21.864969 4746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:21.866137 4746 hybrid_clock.cc:648] HybridClock initialized: now 1781888061866118 us; error 38 us; skew 500 ppm
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:21.868594 4746 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:54:21.869102 4746 webserver.cc:460] Webserver started at http://127.30.148.131:40855/ using document root <none> and password file <none>
I20260619 16:54:21.869324 4746 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:21.869414 4746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:21.869550 4746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:21.870580 4746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "e2e7d66ea85c49c7be9b55e91419748b"
format_stamp: "Formatted at 2026-06-19 16:54:21 on dist-test-slave-xb9n"
server_key: "c0360d78712d9bfbf20c66d6127f4ad4"
server_key_iv: "06ca9890ce78eac4052b3b31a72504c8"
server_key_version: "encryptionkey@0"
I20260619 16:54:21.870939 4746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "e2e7d66ea85c49c7be9b55e91419748b"
format_stamp: "Formatted at 2026-06-19 16:54:21 on dist-test-slave-xb9n"
server_key: "c0360d78712d9bfbf20c66d6127f4ad4"
server_key_iv: "06ca9890ce78eac4052b3b31a72504c8"
server_key_version: "encryptionkey@0"
I20260619 16:54:21.872213 4746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:54:21.872908 4761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:21.873080 4746 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:21.873173 4746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "e2e7d66ea85c49c7be9b55e91419748b"
format_stamp: "Formatted at 2026-06-19 16:54:21 on dist-test-slave-xb9n"
server_key: "c0360d78712d9bfbf20c66d6127f4ad4"
server_key_iv: "06ca9890ce78eac4052b3b31a72504c8"
server_key_version: "encryptionkey@0"
I20260619 16:54:21.873257 4746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:21.886894 4746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:21.888867 4746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:21.889029 4746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:21.889235 4746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:21.889631 4746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:21.889684 4746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:21.889725 4746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:21.889752 4746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:21.895519 4746 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:43415
I20260619 16:54:21.895556 4874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:43415 every 8 connection(s)
I20260619 16:54:21.895869 4746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:54:21.897811 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 4746
I20260619 16:54:21.897904 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:21.898130 31314 external_mini_cluster.cc:1428] Setting key ea1c27525b07b1d1d8264cfc385560fe
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:21.904989 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:21.896564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:45205 (local address 127.30.148.190:41071)
0619 16:54:21.896608 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:21.896610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:21.896864 (+ 254us) server_negotiation.cc:407] Connection header received
0619 16:54:21.898059 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:21.898060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:21.898071 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:21.898127 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:21.898870 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.899458 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.900397 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.900582 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.900632 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:21.902682 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:21.902683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:21.902684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:21.902690 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:21.903899 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.904249 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.904250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.904250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.904297 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.904504 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.904505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.904505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.904608 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:21.904657 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:21.904875 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 16:54:21.904927 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:54:21.905135 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:21.896541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41071 (local address 127.30.148.131:45205)
0619 16:54:21.896710 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:21.896773 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:21.897912 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:21.898211 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:21.898217 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:21.898493 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:21.898748 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:21.898769 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.899551 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.899552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:21.900323 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:21.900325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.900656 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.900657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:21.900771 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:21.901196 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:21.901204 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:21.902580 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:21.903969 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:21.903974 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:21.903976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:21.904185 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:21.904353 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:21.904353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:21.904353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:21.904451 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:21.904661 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:21.904664 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:21.904772 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 16:54:21.904850 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 16:54:21.904957 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260619 16:54:21.905581 4875 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41071
I20260619 16:54:21.905669 4875 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:21.905810 4875 heartbeater.cc:511] Master 127.30.148.190:41071 requested a full tablet report, sending...
I20260619 16:54:21.906167 4416 ts_manager.cc:194] Registered new tserver with Master: e2e7d66ea85c49c7be9b55e91419748b (127.30.148.131:43415)
I20260619 16:54:21.906503 4416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:45205
I20260619 16:54:21.910084 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/test-admin.keytab.
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:21.925203 31314 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 16:54:21.926767 4882 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:21.926038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56456 (local address 127.30.148.129:34667)
0619 16:54:21.926195 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:21.926200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:21.926224 (+ 24us) server_negotiation.cc:407] Connection header received
0619 16:54:21.926249 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:21.926251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:21.926264 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:21.926331 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:21.926673 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56456: BlockingRecv error: recv got EOF from 127.0.0.1:56456 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
W20260619 16:54:21.926833 31314 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/krb5cc' of type FILE
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:21.927798 31314 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:21.928630 31314 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.129@KRBTEST.COM
I20260619 16:54:21.936075 4882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:21.929113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56462 (local address 127.30.148.129:34667)
0619 16:54:21.929165 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:21.929167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:21.929188 (+ 21us) server_negotiation.cc:407] Connection header received
0619 16:54:21.929247 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:21.929248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:21.929257 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:21.929303 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:21.929751 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.930451 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.931134 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.931455 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.931512 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:21.933508 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:21.933515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:21.933515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:21.933521 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:21.935104 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.935394 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.935397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.935397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.935449 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.935595 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.935595 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.935596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.935703 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:21.935797 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:21.935858 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 16:54:21.935916 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 16:54:21.937903 4882 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:21.937321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56468 (local address 127.30.148.129:34667)
0619 16:54:21.937388 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:21.937390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:21.937444 (+ 54us) server_negotiation.cc:407] Connection header received
0619 16:54:21.937516 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:21.937517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:21.937527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:21.937580 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:21.937850 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56468: BlockingRecv error: recv got EOF from 127.0.0.1:56468 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 16:54:21.937956 31314 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/krb5cc' of type FILE
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:21.938835 31314 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:21.939643 31314 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.129@KRBTEST.COM
I20260619 16:54:21.945950 4882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:21.940065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56480 (local address 127.30.148.129:34667)
0619 16:54:21.940135 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:21.940137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:21.940191 (+ 54us) server_negotiation.cc:407] Connection header received
0619 16:54:21.940224 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:21.940224 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:21.940232 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:21.940274 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:21.940565 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.941008 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.941750 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.941948 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.941999 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:21.943879 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:21.943880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:21.943881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:21.943886 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:21.945009 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.945302 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.945303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.945303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.945347 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.945539 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.945539 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.945540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.945616 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:21.945678 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:21.945761 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 16:54:21.945814 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260619 16:54:21.947316 4882 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:21.946814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56494 (local address 127.30.148.129:34667)
0619 16:54:21.946863 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:21.946865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:21.946940 (+ 75us) server_negotiation.cc:407] Connection header received
0619 16:54:21.946976 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:21.946976 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:21.946983 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:21.947025 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:21.947272 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56494: BlockingRecv error: recv got EOF from 127.0.0.1:56494 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 16:54:21.947394 31314 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888036930978-31314-0/krb5kdc/krb5cc' of type FILE
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:21.948299 31314 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:21.949098 31314 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 16:54:21 dist-test-slave-xb9n krb5kdc[4371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.129@KRBTEST.COM
I20260619 16:54:21.955610 4882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:21.949603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.30.148.129:34667)
0619 16:54:21.949660 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:21.949662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:21.949710 (+ 48us) server_negotiation.cc:407] Connection header received
0619 16:54:21.949780 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:21.949780 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:21.949789 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:21.949842 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:21.950206 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.950656 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.951329 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:21.951511 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:21.951599 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:21.953485 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:21.953486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:21.953486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:21.953492 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:21.954813 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.955092 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.955093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.955093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.955140 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:21.955281 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:21.955281 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:21.955281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:21.955352 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:21.955412 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:21.955508 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 16:54:21.955557 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 16:54:21.956151 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 4478
I20260619 16:54:21.961009 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 4612
I20260619 16:54:21.965160 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 4746
I20260619 16:54:21.969203 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 4384
2026-06-19T16:54:21Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1805 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:22 dist-test-slave-xb9n krb5kdc[4895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:22 dist-test-slave-xb9n krb5kdc[4895](info): set up 2 sockets
Jun 19 16:54:22 dist-test-slave-xb9n krb5kdc[4895](info): commencing operation
krb5kdc: starting...
W20260619 16:54:23.001556 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:23Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.190.keytab.
Entry for principal oryx/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.190.keytab.
I20260619 16:54:23.110090 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33303
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.190.keytab
--principal=oryx/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:33303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:23.187860 4908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:23.188071 4908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:23.188107 4908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:23.189246 4908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:23.189313 4908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:23.189337 4908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:23.189359 4908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:23.189414 4908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:23.191243 4908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:33303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.190.keytab
--principal=oryx/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:23.191535 4908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:23.191751 4908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:23.196273 4914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:23.196273 4913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:23.196398 4908 server_base.cc:1034] running on GCE node
W20260619 16:54:23.196465 4916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:23.196674 4908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:23.196915 4908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:23.198081 4908 hybrid_clock.cc:648] HybridClock initialized: now 1781888063198070 us; error 29 us; skew 500 ppm
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:23.203038 4908 init.cc:377] Logged in from keytab as oryx/127.30.148.190@KRBTEST.COM (short username oryx)
I20260619 16:54:23.203384 4908 webserver.cc:460] Webserver started at http://127.30.148.190:39831/ using document root <none> and password file <none>
I20260619 16:54:23.203570 4908 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:23.203609 4908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:23.203709 4908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:23.204708 4908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "db5925651cfe41e4840bd653b1642f06"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "e05429aeeac7ae5fbb1bb0bf0fc32036"
server_key_iv: "7ee801960794140a09a2b7dc14674fb1"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.204994 4908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "db5925651cfe41e4840bd653b1642f06"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "e05429aeeac7ae5fbb1bb0bf0fc32036"
server_key_iv: "7ee801960794140a09a2b7dc14674fb1"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.206266 4908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:23.206933 4923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.207118 4908 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:23.207196 4908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "db5925651cfe41e4840bd653b1642f06"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "e05429aeeac7ae5fbb1bb0bf0fc32036"
server_key_iv: "7ee801960794140a09a2b7dc14674fb1"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.207247 4908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:23.214099 4908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:23.215721 4908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:23.215821 4908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:23.219453 4908 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:33303
I20260619 16:54:23.219516 4975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:33303 every 8 connection(s)
I20260619 16:54:23.219826 4908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:23.220762 4976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:23.222875 4976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06: Bootstrap starting.
I20260619 16:54:23.223410 4976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:23.223690 4976 log.cc:824] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:23.224205 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 4908
I20260619 16:54:23.224315 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:23.224411 4976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06: No bootstrap required, opened a new log
I20260619 16:54:23.224506 31314 external_mini_cluster.cc:1428] Setting key ca7e0384c0ed847591319a9525e90a1c
I20260619 16:54:23.226305 4976 raft_consensus.cc:348] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } }
I20260619 16:54:23.226449 4976 raft_consensus.cc:374] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:23.226470 4976 raft_consensus.cc:729] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5925651cfe41e4840bd653b1642f06, State: Initialized, Role: FOLLOWER
I20260619 16:54:23.226627 4976 consensus_queue.cc:260] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } }
I20260619 16:54:23.226687 4976 raft_consensus.cc:388] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:23.226707 4976 raft_consensus.cc:482] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:23.226775 4976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:23.227286 4976 raft_consensus.cc:504] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } }
I20260619 16:54:23.227365 4976 leader_election.cc:302] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db5925651cfe41e4840bd653b1642f06; no voters:
I20260619 16:54:23.227612 4976 leader_election.cc:288] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:23.227636 4981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:23.227742 4981 raft_consensus.cc:686] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 1 LEADER]: Becoming Leader. State: Replica: db5925651cfe41e4840bd653b1642f06, State: Running, Role: LEADER
I20260619 16:54:23.227818 4981 consensus_queue.cc:237] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } }
I20260619 16:54:23.227865 4976 sys_catalog.cc:564] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:23.228104 4981 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } }
I20260619 16:54:23.228173 4981 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:23.228309 4981 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db5925651cfe41e4840bd653b1642f06. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } }
I20260619 16:54:23.228380 4981 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:23.228497 4986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:23.228999 4986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:23.231339 4986 catalog_manager.cc:1347] Generated new cluster ID: 6ebe8142fd404b3380dfaf6332afbf45
I20260619 16:54:23.231381 4986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
I20260619 16:54:23.236292 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.224959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.30.148.190:33303)
0619 16:54:23.225142 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:23.225150 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:23.225184 (+ 34us) server_negotiation.cc:407] Connection header received
0619 16:54:23.225680 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:23.225686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:23.225731 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:23.225866 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:23.226555 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.227952 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.229091 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.229696 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.229774 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.232470 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:23.232473 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:23.232473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:23.232483 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:23.234388 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.234871 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.234874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.234874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.234979 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.235215 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.235216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.235216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.235483 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:23.235634 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:23.235946 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 16:54:23.236083 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260619 16:54:23.246088 4986 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:23.246749 4986 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:23.253194 4986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06: Generated new TSK 0
I20260619 16:54:23.253341 4986 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab.
Entry for principal oryx/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab.
I20260619 16:54:23.280097 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33303
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab
--principal=oryx/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:23.357985 5002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:23.358210 5002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:23.358245 5002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:23.359359 5002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:23.359427 5002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:23.359483 5002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:23.360899 5002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab
--principal=oryx/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:23.361217 5002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:23.361557 5002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:23.366457 5007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:23.366466 5010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:23.366613 5002 server_base.cc:1034] running on GCE node
W20260619 16:54:23.366475 5008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:23.366954 5002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:23.367157 5002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:23.368328 5002 hybrid_clock.cc:648] HybridClock initialized: now 1781888063368316 us; error 28 us; skew 500 ppm
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:23.370816 5002 init.cc:377] Logged in from keytab as oryx/127.30.148.129@KRBTEST.COM (short username oryx)
I20260619 16:54:23.371176 5002 webserver.cc:460] Webserver started at http://127.30.148.129:42151/ using document root <none> and password file <none>
I20260619 16:54:23.371344 5002 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:23.371383 5002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:23.371512 5002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:23.372447 5002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "d8e6aba81fb64f3cb8e52d70d122074e"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "4931847db67ccc19609bfb1cdd4c88d5"
server_key_iv: "5365ebbb3cbb42f96292eb0adcfe9cb7"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.372748 5002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "d8e6aba81fb64f3cb8e52d70d122074e"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "4931847db67ccc19609bfb1cdd4c88d5"
server_key_iv: "5365ebbb3cbb42f96292eb0adcfe9cb7"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.374022 5002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:23.374735 5017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.374883 5002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:23.374965 5002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "d8e6aba81fb64f3cb8e52d70d122074e"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "4931847db67ccc19609bfb1cdd4c88d5"
server_key_iv: "5365ebbb3cbb42f96292eb0adcfe9cb7"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.375020 5002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:23.390275 5002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:23.392033 5002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:23.392138 5002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:23.392299 5002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:23.392513 5002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:23.392549 5002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.392568 5002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:23.392581 5002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.398183 5002 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:44817
I20260619 16:54:23.398238 5130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:44817 every 8 connection(s)
I20260619 16:54:23.398536 5002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
I20260619 16:54:23.404686 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 5002
I20260619 16:54:23.404791 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:23.404983 31314 external_mini_cluster.cc:1428] Setting key 631bae579c56e6334ab1d136f766a2ff
I20260619 16:54:23.407323 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.399189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:52915 (local address 127.30.148.190:33303)
0619 16:54:23.399227 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:23.399229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:23.399454 (+ 225us) server_negotiation.cc:407] Connection header received
0619 16:54:23.399890 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:23.399891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:23.399899 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:23.399958 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:23.400787 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.401265 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.402221 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.402410 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.402462 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.404392 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:23.404393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:23.404393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:23.404400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:23.405835 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.406308 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.406309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.406310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.406372 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.406636 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.406636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.406637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.406727 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:23.406776 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:23.407109 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 16:54:23.407200 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 16:54:23.407460 5133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.399151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.129:52915)
0619 16:54:23.399306 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:23.399342 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:23.399810 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:23.400007 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:23.400010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:23.400537 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:23.400711 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.400716 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.401338 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.401338 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.402150 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.402152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.402479 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.402480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.402571 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.403028 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:23.403036 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:23.404315 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:23.405907 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.405914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.405916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.406233 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.406437 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.406438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.406438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.406575 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.406789 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:23.406794 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:23.406952 (+ 158us) client_negotiation.cc:769] Sending connection context
0619 16:54:23.407062 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 16:54:23.407248 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260619 16:54:23.407965 5131 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33303
I20260619 16:54:23.408051 5131 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:23.408206 5131 heartbeater.cc:511] Master 127.30.148.190:33303 requested a full tablet report, sending...
I20260619 16:54:23.408790 4940 ts_manager.cc:194] Registered new tserver with Master: d8e6aba81fb64f3cb8e52d70d122074e (127.30.148.129:44817)
I20260619 16:54:23.409756 4940 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:52915
WARNING: no policy specified for oryx/127.30.148.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab.
Entry for principal oryx/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab.
I20260619 16:54:23.427809 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33303
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab
--principal=oryx/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:23.505617 5136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:23.505805 5136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:23.505827 5136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:23.506881 5136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:23.506915 5136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:23.506942 5136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:23.508272 5136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab
--principal=oryx/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:23.508438 5136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:23.508595 5136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:23.513296 5142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:23.513307 5141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:23.513307 5144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:23.513476 5136 server_base.cc:1034] running on GCE node
I20260619 16:54:23.513700 5136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:23.513948 5136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:23.515092 5136 hybrid_clock.cc:648] HybridClock initialized: now 1781888063515069 us; error 36 us; skew 500 ppm
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:23.517341 5136 init.cc:377] Logged in from keytab as oryx/127.30.148.130@KRBTEST.COM (short username oryx)
I20260619 16:54:23.517849 5136 webserver.cc:460] Webserver started at http://127.30.148.130:46177/ using document root <none> and password file <none>
I20260619 16:54:23.518076 5136 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:23.518143 5136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:23.518289 5136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:23.519279 5136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "fb4bedcc98da432f8244bac04ed47c42"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "b0f6a0bcef16be1787b534c6ee939d54"
server_key_iv: "e24a62213c98aa28549e62c17c9e2b7e"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.519632 5136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "fb4bedcc98da432f8244bac04ed47c42"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "b0f6a0bcef16be1787b534c6ee939d54"
server_key_iv: "e24a62213c98aa28549e62c17c9e2b7e"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.520900 5136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:54:23.521713 5151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.521893 5136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:23.521986 5136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "fb4bedcc98da432f8244bac04ed47c42"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "b0f6a0bcef16be1787b534c6ee939d54"
server_key_iv: "e24a62213c98aa28549e62c17c9e2b7e"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.522065 5136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:23.532402 5136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:23.534142 5136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:23.534296 5136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:23.534502 5136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:23.534797 5136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:23.534852 5136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.534883 5136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:23.534910 5136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.540628 5136 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:38313
I20260619 16:54:23.540673 5264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:38313 every 8 connection(s)
I20260619 16:54:23.540971 5136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:54:23.542027 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 5136
I20260619 16:54:23.542138 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:23.542330 31314 external_mini_cluster.cc:1428] Setting key 9adc8a96c53c943dad9f1eecc4b9b77e
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.30.148.131@KRBTEST.COM; defaulting to no policy
I20260619 16:54:23.551095 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.541646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:44571 (local address 127.30.148.190:33303)
0619 16:54:23.541719 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:23.541721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:23.541921 (+ 200us) server_negotiation.cc:407] Connection header received
0619 16:54:23.542615 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:23.542616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:23.542627 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:23.542679 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:23.543902 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.544385 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.545453 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.545667 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.545772 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.548145 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:23.548146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:23.548147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:23.548155 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:23.549722 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.550092 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.550093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.550094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.550155 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.550391 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.550392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.550392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.550529 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:23.550609 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:23.550915 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 16:54:23.550994 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 16:54:23.551271 5267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.541636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.130:44571)
0619 16:54:23.541790 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:23.541828 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:23.542492 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:23.542759 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:23.542763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:23.543534 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:23.543797 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.543803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.544488 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.544489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.545316 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.545318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.545779 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.545780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.545914 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.546449 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:23.546457 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:23.548049 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:23.549796 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.549801 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.549803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.550013 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.550233 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.550234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.550234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.550324 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.550603 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:23.550606 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:23.550708 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 16:54:23.550884 (+ 176us) client_negotiation.cc:240] Negotiation successful
0619 16:54:23.551035 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260619 16:54:23.551805 5265 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33303
I20260619 16:54:23.551877 5265 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:23.552004 5265 heartbeater.cc:511] Master 127.30.148.190:33303 requested a full tablet report, sending...
I20260619 16:54:23.552341 4940 ts_manager.cc:194] Registered new tserver with Master: fb4bedcc98da432f8244bac04ed47c42 (127.30.148.130:38313)
I20260619 16:54:23.552690 4940 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:44571
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab.
Entry for principal oryx/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab.
I20260619 16:54:23.565855 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33303
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab
--principal=oryx/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:23.644306 5270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:23.644482 5270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:23.644503 5270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:23.645547 5270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:23.645578 5270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:23.645604 5270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:23.646919 5270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab
--principal=oryx/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:23.647087 5270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:23.647243 5270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:23.651876 5275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:23.651885 5278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:23.651885 5276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:23.651968 5270 server_base.cc:1034] running on GCE node
I20260619 16:54:23.652324 5270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:23.652565 5270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:23.653720 5270 hybrid_clock.cc:648] HybridClock initialized: now 1781888063653713 us; error 37 us; skew 500 ppm
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:23.656078 5270 init.cc:377] Logged in from keytab as oryx/127.30.148.131@KRBTEST.COM (short username oryx)
I20260619 16:54:23.656492 5270 webserver.cc:460] Webserver started at http://127.30.148.131:35073/ using document root <none> and password file <none>
I20260619 16:54:23.656710 5270 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:23.656780 5270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:23.656926 5270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:23.657954 5270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "9f3a049c850b48bd8bf962e2a9108377"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "1a8f0af48552916147a8d369182bc73f"
server_key_iv: "2a0181e8fc165f420da392859328e879"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.658295 5270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "9f3a049c850b48bd8bf962e2a9108377"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "1a8f0af48552916147a8d369182bc73f"
server_key_iv: "2a0181e8fc165f420da392859328e879"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.659533 5270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:23.660202 5285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.660374 5270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:23.660465 5270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "9f3a049c850b48bd8bf962e2a9108377"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "1a8f0af48552916147a8d369182bc73f"
server_key_iv: "2a0181e8fc165f420da392859328e879"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.660547 5270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:23.694191 5270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:23.696467 5270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:23.696627 5270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:23.696849 5270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:23.697147 5270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:23.697206 5270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.697254 5270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:23.697285 5270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.702900 5270 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:37105
I20260619 16:54:23.702948 5398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:37105 every 8 connection(s)
I20260619 16:54:23.703243 5270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
I20260619 16:54:23.710062 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 5270
I20260619 16:54:23.710196 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:23.710402 31314 external_mini_cluster.cc:1428] Setting key 30a520deaf78bb4b6d82f9433201ed15
I20260619 16:54:23.712222 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.703878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:47113 (local address 127.30.148.190:33303)
0619 16:54:23.703958 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:23.703960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:23.704158 (+ 198us) server_negotiation.cc:407] Connection header received
0619 16:54:23.704660 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:23.704661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:23.704672 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:23.704731 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:23.705621 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.706122 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.707127 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.707323 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.707409 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.709340 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:23.709341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:23.709341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:23.709349 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:23.711022 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.711486 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.711487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.711487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.711546 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.711731 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.711731 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.711732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.711828 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:23.711883 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:23.712093 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 16:54:23.712151 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 16:54:23.712358 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.703851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.131:47113)
0619 16:54:23.704037 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:23.704070 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:23.704553 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:23.704803 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:23.704807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:23.705324 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:23.705512 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.705517 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.706226 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.706227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.707035 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.707037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.707419 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.707419 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.707557 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.707967 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:23.707975 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:23.709240 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:23.711113 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.711119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.711121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.711415 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.711588 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.711589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.711589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.711681 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.711872 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:23.711874 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:23.711992 (+ 118us) client_negotiation.cc:769] Sending connection context
0619 16:54:23.712068 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 16:54:23.712176 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260619 16:54:23.712841 5399 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33303
I20260619 16:54:23.712913 5399 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:23.713092 5399 heartbeater.cc:511] Master 127.30.148.190:33303 requested a full tablet report, sending...
I20260619 16:54:23.713503 4940 ts_manager.cc:194] Registered new tserver with Master: 9f3a049c850b48bd8bf962e2a9108377 (127.30.148.131:37105)
I20260619 16:54:23.713828 4940 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:47113
I20260619 16:54:23.722090 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.148.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.189.keytab.
Entry for principal oryx/127.30.148.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.189.keytab.
I20260619 16:54:23.745229 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.189:35425
--webserver_interface=127.30.148.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.189.keytab
--principal=oryx/127.30.148.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:33303,127.30.148.189:35425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:23.824646 5404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:23.824910 5404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:23.824936 5404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:23.826090 5404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:23.826130 5404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:23.826145 5404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:23.826157 5404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:23.826170 5404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:23.827628 5404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:33303,127.30.148.189:35425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.189.keytab
--principal=oryx/127.30.148.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.189:35425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.148.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
W20260619 16:54:23.827770 5404 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.148.190:33303,127.30.148.189:35425'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 16:54:23.827891 5404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:23.828055 5404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:23.832815 5410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:23.832854 5409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:23.832841 5404 server_base.cc:1034] running on GCE node
W20260619 16:54:23.832958 5412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:23.833221 5404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:23.833510 5404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:23.834646 5404 hybrid_clock.cc:648] HybridClock initialized: now 1781888063834627 us; error 37 us; skew 500 ppm
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:23.837026 5404 init.cc:377] Logged in from keytab as oryx/127.30.148.189@KRBTEST.COM (short username oryx)
I20260619 16:54:23.837517 5404 webserver.cc:460] Webserver started at http://127.30.148.189:37903/ using document root <none> and password file <none>
I20260619 16:54:23.837707 5404 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:23.837756 5404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:23.837844 5404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:23.838752 5404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data/instance:
uuid: "256dd41ec994418b9fd88957eeb58a5a"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "9ba28ae2469262fe68dd3ffdcf3d8c71"
server_key_iv: "7a6d36c5b7030613e919b3184607c5e9"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.839027 5404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/wal/instance:
uuid: "256dd41ec994418b9fd88957eeb58a5a"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "9ba28ae2469262fe68dd3ffdcf3d8c71"
server_key_iv: "7a6d36c5b7030613e919b3184607c5e9"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.840217 5404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:54:23.840853 5419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.841018 5404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:23.841111 5404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/wal
uuid: "256dd41ec994418b9fd88957eeb58a5a"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "9ba28ae2469262fe68dd3ffdcf3d8c71"
server_key_iv: "7a6d36c5b7030613e919b3184607c5e9"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.841188 5404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:23.848131 5404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:23.850355 5404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:23.850507 5404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:23.854185 5404 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.189:35425
I20260619 16:54:23.854216 5471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.189:35425 every 8 connection(s)
I20260619 16:54:23.854545 5404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
I20260619 16:54:23.855417 5472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:23.856854 5472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:54:23.859133 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 5404
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.189@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.148.189@KRBTEST.COM
I20260619 16:54:23.866309 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.857599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49656 (local address 127.30.148.190:33303)
0619 16:54:23.857661 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:23.857664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:23.857852 (+ 188us) server_negotiation.cc:407] Connection header received
0619 16:54:23.858334 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:23.858336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:23.858349 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:23.858395 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:23.858954 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.859773 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.860973 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.861154 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.861207 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.863452 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:23.863453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:23.863454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:23.863460 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:23.864924 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.865351 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.865352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.865352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.865430 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.865664 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.865666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.865666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.865814 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:23.865893 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:23.866149 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 16:54:23.866224 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 16:54:23.866444 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.857583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:49656)
0619 16:54:23.857742 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:23.857773 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:23.858223 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:23.858469 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:23.858472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:23.858691 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:23.858862 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.858865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.859856 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.859857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.860891 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.860893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.861241 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.861242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.861386 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.861878 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:23.861885 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:23.863376 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:23.865000 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.865006 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.865008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.865285 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.865503 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.865503 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.865504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.865593 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.865885 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:23.865890 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:23.866012 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 16:54:23.866111 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 16:54:23.866241 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260619 16:54:23.867017 5472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:54:23.868163 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.859621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35610 (local address 127.30.148.189:35425)
0619 16:54:23.859762 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:23.859767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:23.859826 (+ 59us) server_negotiation.cc:407] Connection header received
0619 16:54:23.859892 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:23.859894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:23.859914 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:23.859991 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:23.860516 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.861472 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.862288 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.862617 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.862674 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.865277 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:23.865283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:23.865283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:23.865289 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:23.867004 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.867388 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.867391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.867392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.867449 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.867664 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.867665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.867665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.867822 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:23.867957 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:23.868046 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 16:54:23.868100 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":100,"threads_started":1}
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.189@KRBTEST.COM for oryx/127.30.148.189@KRBTEST.COM
I20260619 16:54:23.873322 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.867266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:35425 (local address 127.0.0.1:35620)
0619 16:54:23.867322 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:23.867331 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:23.867367 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:23.867597 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:23.867598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:23.867750 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:23.867799 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.867801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.868683 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.868684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.869208 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.869218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.869610 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.869611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.869661 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.870061 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:23.870065 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:23.871075 (+ 1010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:23.872625 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.872625 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.872626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.872773 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.872932 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.872933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.872933 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.872978 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.873133 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:23.873133 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:23.873161 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 16:54:23.873204 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 16:54:23.873270 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 16:54:23.873484 5477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.867346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.30.148.189:35425)
0619 16:54:23.867456 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:23.867457 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:23.867464 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:54:23.867495 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:23.867496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:23.867507 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:23.867579 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:23.867888 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.868601 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.869283 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.869546 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.869599 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.871144 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:23.871145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:23.871145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:23.871151 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:23.872561 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.872833 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.872834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.872834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.872879 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.873014 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.873015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.873015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.873089 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:23.873131 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:23.873218 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 16:54:23.873310 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":91,"thread_start_us":69,"threads_started":1}
I20260619 16:54:23.874547 5472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: Bootstrap starting.
I20260619 16:54:23.875138 5472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:23.875353 5472 log.cc:824] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:23.875960 5472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: No bootstrap required, opened a new log
I20260619 16:54:23.877243 5472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } }
I20260619 16:54:23.877400 5472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:23.877444 5472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256dd41ec994418b9fd88957eeb58a5a, State: Initialized, Role: FOLLOWER
I20260619 16:54:23.877540 5472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } }
I20260619 16:54:23.877769 5479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } } }
I20260619 16:54:23.877840 5472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:23.877852 5479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: This master's current role is: FOLLOWER
W20260619 16:54:23.879434 5490 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:54:23.879489 5490 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:54:23.880823 5404 master_runner.cc:428] Detected that this master 256dd41ec994418b9fd88957eeb58a5a is joining an existing cluster
I20260619 16:54:23.880861 5404 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.148.189:35425
I20260619 16:54:23.888384 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.881830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:35425 (local address 127.0.0.1:35624)
0619 16:54:23.882033 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:23.882043 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:23.882108 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:23.882215 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:23.882216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:23.882436 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:23.882497 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.882499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.883378 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.883378 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.883959 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.883960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.884352 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.884352 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.884418 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.884836 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:23.884841 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:23.885212 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:23.887175 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.887176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.887177 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.887341 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.887556 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.887557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.887558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.887611 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.887902 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:23.887902 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:23.887948 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 16:54:23.888019 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 16:54:23.888108 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260619 16:54:23.888479 5477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.881852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35624 (local address 127.30.148.189:35425)
0619 16:54:23.881888 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:23.881890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:23.882095 (+ 205us) server_negotiation.cc:407] Connection header received
0619 16:54:23.882155 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:23.882156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:23.882163 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:23.882218 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:23.882613 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.883299 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.884041 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.884268 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.884350 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.885294 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:23.885295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:23.885295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:23.885302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:23.887074 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.887424 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.887425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.887425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.887489 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.887690 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.887690 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.887691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.887824 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:23.887882 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:23.888282 (+ 400us) server_negotiation.cc:299] Negotiation successful
0619 16:54:23.888358 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:54:23.888577 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.882028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49672 (local address 127.30.148.190:33303)
0619 16:54:23.882069 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:23.882071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:23.882078 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:54:23.882098 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:23.882099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:23.882109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:23.882156 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:23.882529 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.883283 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.884380 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.884688 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.884729 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.885789 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:23.885790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:23.885790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:23.885797 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:23.887351 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.887748 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.887749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.887749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.887796 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.888031 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.888031 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.888032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.888157 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:23.888223 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:23.888402 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 16:54:23.888479 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:54:23.889101 5497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.881794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:49672)
0619 16:54:23.881951 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:23.881960 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:23.882029 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:23.882195 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:23.882196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:23.882396 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:23.882451 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.882452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.883378 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.883379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.884275 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.884278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.884802 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.884803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.884867 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.885259 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:23.885264 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:23.885712 (+ 448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:23.887443 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.887444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.887445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.887680 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.887872 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.887873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.887873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.887949 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.888231 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:23.888232 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:23.888274 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 16:54:23.888908 (+ 634us) client_negotiation.cc:240] Negotiation successful
0619 16:54:23.889004 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":29,"thread_start_us":88,"threads_started":1}
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.190@KRBTEST.COM for oryx/127.30.148.189@KRBTEST.COM
I20260619 16:54:23.898093 5477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.891333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35638 (local address 127.30.148.189:35425)
0619 16:54:23.891367 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:23.891369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:23.891377 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:23.891447 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:23.891448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:23.891457 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:23.891523 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:23.891905 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.892509 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.893269 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.893534 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.893615 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.895592 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:23.895593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:23.895594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:23.895600 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:23.896904 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.897304 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.897305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.897305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.897388 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.897568 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.897569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.897570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.897672 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:23.897760 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:23.897935 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 16:54:23.898007 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:54:23.898121 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.891146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:35425 (local address 127.0.0.1:35638)
0619 16:54:23.891277 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:23.891318 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:23.891382 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:23.891529 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:23.891532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:23.891768 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:23.891806 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.891808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.892591 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.892592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.893148 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.893149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.893620 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.893621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.893696 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.894142 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:23.894146 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:23.895505 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:23.897002 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.897009 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.897011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.897218 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.897439 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.897440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.897440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.897498 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.897757 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:23.897764 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:23.897811 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 16:54:23.897909 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 16:54:23.898010 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260619 16:54:23.898783 4940 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.148.189:35425: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a"
member_type: NON_VOTER
last_known_addr {
host: "127.30.148.189"
port: 35425
}
attrs {
promote: true
}
}
dest_uuid: "db5925651cfe41e4840bd653b1642f06"
cas_config_opid_index: -1
I20260619 16:54:23.899070 4940 consensus_queue.cc:237] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true } }
I20260619 16:54:23.899501 5500 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 256dd41ec994418b9fd88957eeb58a5a (127.30.148.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true } } }
I20260619 16:54:23.899838 5446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:23.900579 5500 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.148.189:35425
I20260619 16:54:23.900708 5446 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 FOLLOWER]: Refusing update from remote peer db5925651cfe41e4840bd653b1642f06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 16:54:23.900795 5404 master.cc:537] Master@127.30.148.189:35425 shutting down...
I20260619 16:54:23.901149 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true } } }
I20260619 16:54:23.901324 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:23.901209 5500 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 16:54:23.902547 4927 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 16:54:23.903029 4927 consensus_peers.cc:487] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 -> Peer 256dd41ec994418b9fd88957eeb58a5a (127.30.148.189:35425): Couldn't send request to peer 256dd41ec994418b9fd88957eeb58a5a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 16:54:23.903323 5404 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:23.903412 5404 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:23.903460 5404 tablet_replica.cc:330] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: stopping tablet replica
I20260619 16:54:23.915679 5404 master.cc:552] Master@127.30.148.189:35425 shutdown complete.
I20260619 16:54:23.915724 5404 master_runner.cc:305] Clearing existing system tablet
I20260619 16:54:23.915860 5404 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:23.915899 5404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:23.916469 5404 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:23.916939 5504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.917044 5404 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:23.917099 5404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.917124 5404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/wal
uuid: "256dd41ec994418b9fd88957eeb58a5a"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "9ba28ae2469262fe68dd3ffdcf3d8c71"
server_key_iv: "7a6d36c5b7030613e919b3184607c5e9"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.918179 5404 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:23.919339 5404 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 16:54:23.919418 5404 log.cc:1196] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 16:54:23.919603 5404 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: Deleting consensus metadata
I20260619 16:54:23.919736 5404 master_runner.cc:315] Copying system tablet from 127.30.148.190:33303
I20260619 16:54:23.920302 5404 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: tablet copy: Beginning tablet copy session from remote peer at address 127.30.148.190:33303
I20260619 16:54:23.926294 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.920635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49680 (local address 127.30.148.190:33303)
0619 16:54:23.920679 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:23.920681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:23.920732 (+ 51us) server_negotiation.cc:407] Connection header received
0619 16:54:23.920792 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:23.920792 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:23.920799 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:23.920841 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:23.921158 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.921682 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.922705 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.922880 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.922925 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.923865 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:23.923866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:23.923866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:23.923872 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:23.925264 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.925652 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.925653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.925653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.925701 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.925903 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.925904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.925904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.925991 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:23.926039 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:23.926185 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 16:54:23.926240 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:54:23.926473 5510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.920549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:49680)
0619 16:54:23.920663 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:23.920673 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:23.920734 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:23.920855 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:23.920856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:23.921033 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:23.921091 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.921093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.921767 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.921768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.922624 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:23.922626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.922946 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.922947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:23.923012 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.923395 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:23.923399 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:23.923788 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:23.925334 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.925335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.925336 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.925575 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.925768 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:23.925769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:23.925769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:23.925830 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:23.926052 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:23.926053 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:23.926093 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 16:54:23.926169 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 16:54:23.926262 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":76,"threads_started":1}
I20260619 16:54:23.926884 4960 tablet_copy_service.cc:140] P db5925651cfe41e4840bd653b1642f06: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 256dd41ec994418b9fd88957eeb58a5a ({username='oryx', principal='oryx/127.30.148.189@KRBTEST.COM'} at 127.0.0.1:49680)
I20260619 16:54:23.926973 4960 tablet_copy_service.cc:161] P db5925651cfe41e4840bd653b1642f06: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 256dd41ec994418b9fd88957eeb58a5a at {username='oryx', principal='oryx/127.30.148.189@KRBTEST.COM'} at 127.0.0.1:49680: session id = 256dd41ec994418b9fd88957eeb58a5a-00000000000000000000000000000000
I20260619 16:54:23.927572 4960 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 16:54:23.928161 5404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 16:54:23.929059 5404 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: tablet copy: Starting download of 0 data blocks...
I20260619 16:54:23.929208 5404 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: tablet copy: Starting download of 1 WAL segments...
I20260619 16:54:23.930073 5404 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 16:54:23.930795 4960 tablet_copy_service.cc:342] P db5925651cfe41e4840bd653b1642f06: Request end of tablet copy session 256dd41ec994418b9fd88957eeb58a5a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.148.189@KRBTEST.COM'} at 127.0.0.1:49680
I20260619 16:54:23.930866 4960 tablet_copy_service.cc:433] P db5925651cfe41e4840bd653b1642f06: ending tablet copy session 256dd41ec994418b9fd88957eeb58a5a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 256dd41ec994418b9fd88957eeb58a5a
I20260619 16:54:23.933233 5404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:23.934423 5511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:23.934474 5512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:23.934496 5514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:23.934580 5404 server_base.cc:1034] running on GCE node
I20260619 16:54:23.934679 5404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:23.934834 5404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:23.935945 5404 hybrid_clock.cc:648] HybridClock initialized: now 1781888063935936 us; error 23 us; skew 500 ppm
Jun 19 16:54:23 dist-test-slave-xb9n krb5kdc[4895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:23.937294 5404 init.cc:377] Logged in from keytab as oryx/127.30.148.189@KRBTEST.COM (short username oryx)
I20260619 16:54:23.937592 5404 webserver.cc:460] Webserver started at http://127.30.148.189:33375/ using document root <none> and password file <none>
I20260619 16:54:23.937732 5404 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:23.937779 5404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:23.938338 5404 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:23.938824 5521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.938970 5404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:23.939024 5404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/wal
uuid: "256dd41ec994418b9fd88957eeb58a5a"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "9ba28ae2469262fe68dd3ffdcf3d8c71"
server_key_iv: "7a6d36c5b7030613e919b3184607c5e9"
server_key_version: "encryptionkey@0"
I20260619 16:54:23.939074 5404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:23.956686 5404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:23.958220 5404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:23.961187 5404 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.189:35425
I20260619 16:54:23.961378 5573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.189:35425 every 8 connection(s)
I20260619 16:54:23.961547 5404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
I20260619 16:54:23.962191 5574 sys_catalog.cc:263] Verifying existing consensus state
I20260619 16:54:23.962591 5574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: Bootstrap starting.
I20260619 16:54:23.964794 5574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:54:23.965014 5574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: Bootstrap complete.
I20260619 16:54:23.965205 5574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true } }
I20260619 16:54:23.965294 5574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 256dd41ec994418b9fd88957eeb58a5a, State: Initialized, Role: LEARNER
I20260619 16:54:23.965343 5574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true } }
I20260619 16:54:23.965516 5574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:23.965526 5577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true } } }
I20260619 16:54:23.965672 5577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: This master's current role is: LEARNER
I20260619 16:54:23.966958 5588 catalog_manager.cc:1259] Loaded cluster ID: 6ebe8142fd404b3380dfaf6332afbf45
I20260619 16:54:23.967005 5588 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: loading cluster ID for follower catalog manager: success
I20260619 16:54:23.967989 5588 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: acquiring CA information for follower catalog manager: success
I20260619 16:54:23.968264 5588 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 16:54:23.976492 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:23.969696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.30.148.189:35425)
0619 16:54:23.969852 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:23.969854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:23.969864 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:23.969888 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:23.969889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:23.969900 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:23.969984 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:23.970508 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.971485 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.972232 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:23.972554 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:23.972633 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:23.973855 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:23.973857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:23.973857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:23.973864 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:23.975420 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.975746 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.975747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.975747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.975808 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:23.975994 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:23.975995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:23.975995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:23.976126 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:23.976231 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:23.976312 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 16:54:23.976388 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for oryx/127.30.148.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.188.keytab.
Entry for principal oryx/127.30.148.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.188.keytab.
I20260619 16:54:23.998041 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.188:44279
--webserver_interface=127.30.148.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.188.keytab
--principal=oryx/127.30.148.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:33303,127.30.148.189:35425,127.30.148.188:44279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:24.077243 5592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:24.077531 5592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:24.077572 5592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:24.079173 5592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:24.079237 5592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:24.079265 5592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:24.079295 5592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:24.079321 5592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:24.080771 5592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:33303,127.30.148.189:35425,127.30.148.188:44279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.188.keytab
--principal=oryx/127.30.148.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.188:44279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.148.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:24.081024 5592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:24.081256 5592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:24.085917 5598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:24.085970 5592 server_base.cc:1034] running on GCE node
W20260619 16:54:24.085939 5597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:24.085917 5600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:24.086386 5592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:24.086663 5592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:24.087800 5592 hybrid_clock.cc:648] HybridClock initialized: now 1781888064087782 us; error 52 us; skew 500 ppm
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:24.090231 5592 init.cc:377] Logged in from keytab as oryx/127.30.148.188@KRBTEST.COM (short username oryx)
I20260619 16:54:24.090600 5592 webserver.cc:460] Webserver started at http://127.30.148.188:42457/ using document root <none> and password file <none>
I20260619 16:54:24.090762 5592 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:24.090801 5592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:24.090883 5592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:24.091894 5592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data/instance:
uuid: "8c69a169602348ffb729a47f62e547ad"
format_stamp: "Formatted at 2026-06-19 16:54:24 on dist-test-slave-xb9n"
server_key: "4ee8cc10e80130499967ee2832013e27"
server_key_iv: "7bd1228878ae2288ddc96a70e06dc347"
server_key_version: "encryptionkey@0"
I20260619 16:54:24.092181 5592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/wal/instance:
uuid: "8c69a169602348ffb729a47f62e547ad"
format_stamp: "Formatted at 2026-06-19 16:54:24 on dist-test-slave-xb9n"
server_key: "4ee8cc10e80130499967ee2832013e27"
server_key_iv: "7bd1228878ae2288ddc96a70e06dc347"
server_key_version: "encryptionkey@0"
I20260619 16:54:24.093557 5592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:54:24.094205 5607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.094367 5592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.094448 5592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/wal
uuid: "8c69a169602348ffb729a47f62e547ad"
format_stamp: "Formatted at 2026-06-19 16:54:24 on dist-test-slave-xb9n"
server_key: "4ee8cc10e80130499967ee2832013e27"
server_key_iv: "7bd1228878ae2288ddc96a70e06dc347"
server_key_version: "encryptionkey@0"
I20260619 16:54:24.094499 5592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:24.111189 5592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:24.113435 5592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:24.113538 5592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:24.117089 5592 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.188:44279
I20260619 16:54:24.117163 5659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.188:44279 every 8 connection(s)
I20260619 16:54:24.117524 5592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
I20260619 16:54:24.118490 5660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:24.119915 5660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:54:24.122202 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 5592
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.188@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.148.188@KRBTEST.COM
I20260619 16:54:24.129504 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.120665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49690 (local address 127.30.148.190:33303)
0619 16:54:24.120748 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.120750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.120860 (+ 110us) server_negotiation.cc:407] Connection header received
0619 16:54:24.121271 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.121272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.121280 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.121336 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.121914 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.122452 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.123703 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.123988 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.124059 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.126437 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.126438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.126438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.126444 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.128020 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.128518 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.128519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.128519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.128577 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.128805 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.128806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.128807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.128952 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.129029 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.129267 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.129347 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 16:54:24.129627 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.120562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:49690)
0619 16:54:24.120737 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.120775 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.121192 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.121436 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.121440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.121667 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.121840 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.121844 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.122558 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.122558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.123579 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.123581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.124090 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.124091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.124207 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.124739 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.124749 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.126351 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.128099 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.128106 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.128108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.128430 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.128638 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.128639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.128639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.128732 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.129023 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.129026 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.129121 (+ 95us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.129196 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.129385 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260619 16:54:24.130219 5660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:54:24.132586 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.122743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58802 (local address 127.30.148.188:44279)
0619 16:54:24.122894 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.122898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.122939 (+ 41us) server_negotiation.cc:407] Connection header received
0619 16:54:24.122995 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.122997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.123016 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.123097 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.123608 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.124619 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.125726 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.125998 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.126082 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.129051 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.129057 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.129057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.129063 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.131456 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.131830 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.131832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.131833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.131883 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.132069 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.132069 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.132070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.132227 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.132363 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.132415 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.132487 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.188@KRBTEST.COM for oryx/127.30.148.189@KRBTEST.COM
I20260619 16:54:24.137164 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.130699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:35425 (local address 127.0.0.1:35668)
0619 16:54:24.130869 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.130879 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.130937 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.131069 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.131069 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.131271 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.131304 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.131306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.132165 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.132165 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.132936 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.132938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.133336 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.133337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.133434 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.133755 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.133758 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.134802 (+ 1044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.136423 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.136424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.136424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.136578 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.136749 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.136749 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.136749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.136795 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.136974 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.136975 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.137005 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.137053 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.137111 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260619 16:54:24.137182 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.130795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35668 (local address 127.30.148.189:35425)
0619 16:54:24.130851 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.130853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.130922 (+ 69us) server_negotiation.cc:407] Connection header received
0619 16:54:24.130998 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.130999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.131010 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.131062 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.131376 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.132092 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.133012 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.133272 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.133326 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.134888 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.134889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.134889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.134896 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.136242 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.136647 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.136647 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.136648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.136693 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.136833 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.136833 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.136833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.136927 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.136971 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.137069 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.137126 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 16:54:24.137580 5660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.188@KRBTEST.COM for oryx/127.30.148.188@KRBTEST.COM
I20260619 16:54:24.143407 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.137805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44279 (local address 127.0.0.1:58810)
0619 16:54:24.137850 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.137860 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.137893 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.138009 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.138010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.138130 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.138151 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.138152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.138764 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.138765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.139265 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.139267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.139654 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.139655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.139702 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.140075 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.140079 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.141105 (+ 1026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.142670 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.142671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.142671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.142816 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.142971 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.142971 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.142971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.143017 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.143183 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.143184 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.143227 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.143273 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.143337 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 16:54:24.143461 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.137894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.30.148.188:44279)
0619 16:54:24.137923 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.137925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.137933 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:24.137949 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.137949 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.137957 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.137997 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.138228 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.138677 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.139352 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.139573 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.139654 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.141177 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.141178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.141179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.141184 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.142599 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.142874 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.142875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.142875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.142918 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.143055 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.143055 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.143055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.143137 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.143179 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.143303 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.143377 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":15}
I20260619 16:54:24.144632 5660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: Bootstrap starting.
I20260619 16:54:24.145493 5660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:24.145761 5660 log.cc:824] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:24.146425 5660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: No bootstrap required, opened a new log
I20260619 16:54:24.147746 5660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } }
I20260619 16:54:24.147881 5660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:24.147927 5660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c69a169602348ffb729a47f62e547ad, State: Initialized, Role: FOLLOWER
I20260619 16:54:24.148054 5660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } }
I20260619 16:54:24.148345 5660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:24.148330 5665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } } }
I20260619 16:54:24.148451 5665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: This master's current role is: FOLLOWER
W20260619 16:54:24.149940 5676 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:54:24.149993 5676 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:54:24.152117 5592 master_runner.cc:428] Detected that this master 8c69a169602348ffb729a47f62e547ad is joining an existing cluster
I20260619 16:54:24.152182 5592 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.148.188:44279
I20260619 16:54:24.159648 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.153091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35678 (local address 127.30.148.189:35425)
0619 16:54:24.153130 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.153132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.153548 (+ 416us) server_negotiation.cc:407] Connection header received
0619 16:54:24.153637 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.153638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.153647 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.153676 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.154096 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.154781 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.155759 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.156049 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.156090 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.157137 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.157139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.157139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.157145 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.158545 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.158894 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.158895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.158896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.158945 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.159146 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.159147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.159148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.159254 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.159323 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.159484 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.159556 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:54:24.159677 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.153144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:35425 (local address 127.0.0.1:35678)
0619 16:54:24.153485 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.153494 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.153571 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.153737 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.153738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.153953 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.154006 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.154007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.154858 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.154859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.155661 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.155662 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.156181 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.156181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.156247 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.156721 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.156725 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.157026 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.158642 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.158643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.158643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.158811 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.159018 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.159018 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.159019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.159076 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.159326 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.159327 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.159378 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.159460 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.159559 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":4,"thread_start_us":74,"threads_started":1}
I20260619 16:54:24.177201 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.153090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49694 (local address 127.30.148.190:33303)
0619 16:54:24.153171 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.153173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.153326 (+ 153us) server_negotiation.cc:407] Connection header received
0619 16:54:24.153424 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.153425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.153435 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.153485 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.153883 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.154647 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.155759 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.156059 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.156094 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.174324 (+ 18230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.174326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.174326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.174334 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.175911 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.176330 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.176331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.176331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.176391 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.176580 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.176581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.176582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.176728 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.176813 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.177020 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.177106 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 16:54:24.177251 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.153321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.30.148.188:44279)
0619 16:54:24.153351 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.153370 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.153578 (+ 208us) server_negotiation.cc:407] Connection header received
0619 16:54:24.153659 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.153660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.153667 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.153692 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.154137 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.154819 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.155810 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.156078 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.156116 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.174639 (+ 18523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.174640 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.174641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.174648 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.176148 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.176527 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.176528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.176529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.176573 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.176737 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.176737 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.176738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.176872 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.176933 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.177084 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.177147 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:24.177402 5682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.153318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44279 (local address 127.0.0.1:58818)
0619 16:54:24.153512 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.153522 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.153599 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.153751 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.153752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.153992 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.154059 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.154060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.154905 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.154906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.155709 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.155721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.156180 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.156181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.156233 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.174178 (+ 17945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.174182 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.174546 (+ 364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.176227 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.176227 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.176228 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.176430 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.176633 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.176633 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.176634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.176679 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.176926 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.176926 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.176962 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.177190 (+ 228us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.177268 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":1,"thread_start_us":134,"threads_started":1}
I20260619 16:54:24.177179 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.153038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:49694)
0619 16:54:24.153261 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.153272 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.153351 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.153532 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.153533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.153737 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.153800 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.153802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.154734 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.154735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.155660 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.155662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.156179 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.156180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.156257 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.173928 (+ 17671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.173932 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.174220 (+ 288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.175995 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.175996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.175997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.176259 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.176467 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.176468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.176468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.176525 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.176841 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.176842 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.176883 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.176977 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.177072 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888063, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.190@KRBTEST.COM for oryx/127.30.148.188@KRBTEST.COM
I20260619 16:54:24.183773 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 5002
I20260619 16:54:24.187222 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.179851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58824 (local address 127.30.148.188:44279)
0619 16:54:24.179884 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.179886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.179894 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:24.179965 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.179966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.179974 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.180036 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.180287 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.180786 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.181772 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.181976 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.182079 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.184381 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.184382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.184382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.184388 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.186036 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.186435 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.186436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.186436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.186495 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.186716 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.186717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.186718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.186833 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.186902 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.187063 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.187135 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:24.187283 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.179788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44279 (local address 127.0.0.1:58824)
0619 16:54:24.179843 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.179854 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.179898 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.180041 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.180041 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.180181 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.180214 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.180215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.180867 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.180868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.181682 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.181684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.182080 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.182081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.182141 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.182545 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.182549 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.183857 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.186126 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.186127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.186127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.186354 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.186574 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.186575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.186576 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.186644 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.186902 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.186903 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.186949 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.187030 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.187133 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 16:54:24.187801 4940 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.148.188:44279: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8c69a169602348ffb729a47f62e547ad"
member_type: NON_VOTER
last_known_addr {
host: "127.30.148.188"
port: 44279
}
attrs {
promote: true
}
}
dest_uuid: "db5925651cfe41e4840bd653b1642f06"
cas_config_opid_index: 5
I20260619 16:54:24.188024 4940 consensus_queue.cc:237] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } }
I20260619 16:54:24.188359 5499 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8c69a169602348ffb729a47f62e547ad (127.30.148.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.189579 5499 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.148.188:44279
I20260619 16:54:24.189661 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.189733 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:24.189790 5592 master.cc:537] Master@127.30.148.188:44279 shutting down...
I20260619 16:54:24.190181 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:44817
--local_ip_for_outbound_sockets=127.30.148.129
--tserver_master_addrs=127.30.148.190:33303
--webserver_port=42151
--webserver_interface=127.30.148.129
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab
--principal=oryx/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.148.190:33303,127.30.148.189:35425
--tserver_master_addrs=127.30.148.190:33303,127.30.148.189:35425,127.30.148.188:44279 with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:24.191802 4926 consensus_peers.cc:487] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 -> Peer 8c69a169602348ffb729a47f62e547ad (127.30.148.188:44279): Couldn't send request to peer 8c69a169602348ffb729a47f62e547ad. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 16:54:24.193403 5592 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 0 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:24.193491 5592 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 0 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:24.193567 5592 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: stopping tablet replica
I20260619 16:54:24.193655 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.189810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35690 (local address 127.30.148.189:35425)
0619 16:54:24.190189 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.190191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.190201 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:24.190223 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.190224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.190244 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.190306 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:24.190482 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.191220 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.192587 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.193182 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.193235 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.193309 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.193560 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.193597 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356}
I20260619 16:54:24.193953 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.188403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:35425 (local address 127.0.0.1:35690)
0619 16:54:24.188575 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.188584 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.188615 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.190366 (+ 1751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.190368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.190369 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:24.190405 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.190406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.191318 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.191319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.192481 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.192485 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.193387 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.193388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.193451 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.193471 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.193841 (+ 370us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.193881 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260619 16:54:24.194196 5548 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 LEARNER]: Refusing update from remote peer db5925651cfe41e4840bd653b1642f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 16:54:24.194482 5500 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 16:54:24.195039 5548 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8c69a169602348ffb729a47f62e547ad (127.30.148.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.196290 5687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.196370 5687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: This master's current role is: LEARNER
I20260619 16:54:24.196537 5577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader db5925651cfe41e4840bd653b1642f06. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: NON_VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: true } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.196633 5577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: This master's current role is: LEARNER
I20260619 16:54:24.196980 5500 raft_consensus.cc:1050] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06: attempting to promote NON_VOTER 256dd41ec994418b9fd88957eeb58a5a to VOTER
I20260619 16:54:24.197152 5500 consensus_queue.cc:237] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } }
I20260619 16:54:24.198446 5548 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 LEARNER]: Refusing update from remote peer db5925651cfe41e4840bd653b1642f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 16:54:24.198488 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.198363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44279 (local address 127.0.0.1:58830)
0619 16:54:24.198419 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.198462 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 16:54:24.198556 4926 consensus_peers.cc:487] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 -> Peer 8c69a169602348ffb729a47f62e547ad (127.30.148.188:44279): Couldn't send request to peer 8c69a169602348ffb729a47f62e547ad. Status: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44279: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 16:54:24.198800 5688 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 16:54:24.199510 5499 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 256dd41ec994418b9fd88957eeb58a5a (127.30.148.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.200338 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.200415 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:24.200464 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.200510 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:24.200722 5548 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 256dd41ec994418b9fd88957eeb58a5a (127.30.148.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.201705 5577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.201788 5577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:54:24.209000 5592 master.cc:552] Master@127.30.148.188:44279 shutdown complete.
I20260619 16:54:24.209048 5592 master_runner.cc:305] Clearing existing system tablet
I20260619 16:54:24.209172 5592 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:24.209213 5592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:24.209936 5592 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.210464 5690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.210572 5592 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:24.210633 5592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.210656 5592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/wal
uuid: "8c69a169602348ffb729a47f62e547ad"
format_stamp: "Formatted at 2026-06-19 16:54:24 on dist-test-slave-xb9n"
server_key: "4ee8cc10e80130499967ee2832013e27"
server_key_iv: "7bd1228878ae2288ddc96a70e06dc347"
server_key_version: "encryptionkey@0"
I20260619 16:54:24.211357 5592 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:24.212630 5592 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 16:54:24.212694 5592 log.cc:1196] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 16:54:24.212893 5592 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: Deleting consensus metadata
I20260619 16:54:24.213004 5592 master_runner.cc:315] Copying system tablet from 127.30.148.190:33303
I20260619 16:54:24.213647 5592 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: tablet copy: Beginning tablet copy session from remote peer at address 127.30.148.190:33303
I20260619 16:54:24.222616 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.213944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49704 (local address 127.30.148.190:33303)
0619 16:54:24.213982 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.213985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.214054 (+ 69us) server_negotiation.cc:407] Connection header received
0619 16:54:24.214117 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.214118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.214127 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.214191 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.214504 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.215284 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.216251 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.216553 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.216626 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.217701 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.217703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.217703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.217721 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.219360 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.221786 (+ 2426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.221787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.221787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.221846 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.222057 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.222058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.222058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.222199 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.222274 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.222455 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.222539 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:54:24.222625 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.213913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:49704)
0619 16:54:24.214001 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.214011 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.214067 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.214193 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.214194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.214378 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.214426 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.214428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.215368 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.215369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.216148 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.216151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.216633 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.216634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.216698 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.217200 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.217206 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.217600 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.221448 (+ 3848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.221449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.221449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.221674 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.221920 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.221921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.221921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.221981 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.222286 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.222287 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.222338 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.222419 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.222525 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":43,"threads_started":1}
I20260619 16:54:24.222896 4960 tablet_copy_service.cc:140] P db5925651cfe41e4840bd653b1642f06: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8c69a169602348ffb729a47f62e547ad ({username='oryx', principal='oryx/127.30.148.188@KRBTEST.COM'} at 127.0.0.1:49704)
I20260619 16:54:24.222961 4960 tablet_copy_service.cc:161] P db5925651cfe41e4840bd653b1642f06: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8c69a169602348ffb729a47f62e547ad at {username='oryx', principal='oryx/127.30.148.188@KRBTEST.COM'} at 127.0.0.1:49704: session id = 8c69a169602348ffb729a47f62e547ad-00000000000000000000000000000000
I20260619 16:54:24.223374 4960 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 16:54:24.223852 5592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 16:54:24.225174 5592 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: tablet copy: Starting download of 0 data blocks...
I20260619 16:54:24.225333 5592 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: tablet copy: Starting download of 1 WAL segments...
I20260619 16:54:24.226178 5592 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 16:54:24.227223 4960 tablet_copy_service.cc:342] P db5925651cfe41e4840bd653b1642f06: Request end of tablet copy session 8c69a169602348ffb729a47f62e547ad-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.148.188@KRBTEST.COM'} at 127.0.0.1:49704
I20260619 16:54:24.227273 4960 tablet_copy_service.cc:433] P db5925651cfe41e4840bd653b1642f06: ending tablet copy session 8c69a169602348ffb729a47f62e547ad-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8c69a169602348ffb729a47f62e547ad
I20260619 16:54:24.230217 5592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:24.231398 5697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:24.231464 5698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:24.231938 5700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:24.232082 5592 server_base.cc:1034] running on GCE node
I20260619 16:54:24.232196 5592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:24.232358 5592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:24.233512 5592 hybrid_clock.cc:648] HybridClock initialized: now 1781888064233491 us; error 38 us; skew 500 ppm
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:24.235028 5592 init.cc:377] Logged in from keytab as oryx/127.30.148.188@KRBTEST.COM (short username oryx)
I20260619 16:54:24.235318 5592 webserver.cc:460] Webserver started at http://127.30.148.188:42521/ using document root <none> and password file <none>
I20260619 16:54:24.235467 5592 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:24.235543 5592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:24.236299 5592 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.237005 5707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.237174 5592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.237251 5592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/wal
uuid: "8c69a169602348ffb729a47f62e547ad"
format_stamp: "Formatted at 2026-06-19 16:54:24 on dist-test-slave-xb9n"
server_key: "4ee8cc10e80130499967ee2832013e27"
server_key_iv: "7bd1228878ae2288ddc96a70e06dc347"
server_key_version: "encryptionkey@0"
I20260619 16:54:24.237370 5592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:24.259825 5592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:24.262012 5592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:24.265233 5592 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.188:44279
I20260619 16:54:24.265322 5759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.188:44279 every 8 connection(s)
I20260619 16:54:24.265796 5592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
I20260619 16:54:24.266811 5760 sys_catalog.cc:263] Verifying existing consensus state
I20260619 16:54:24.267378 5760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: Bootstrap starting.
I20260619 16:54:24.271111 5760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:54:24.271314 5760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: Bootstrap complete.
I20260619 16:54:24.271555 5760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } }
I20260619 16:54:24.271631 5760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8c69a169602348ffb729a47f62e547ad, State: Initialized, Role: LEARNER
I20260619 16:54:24.271663 5760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } }
I20260619 16:54:24.271821 5760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:24.271862 5763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.271960 5763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: This master's current role is: LEARNER
I20260619 16:54:24.273247 5774 catalog_manager.cc:1259] Loaded cluster ID: 6ebe8142fd404b3380dfaf6332afbf45
I20260619 16:54:24.273458 5774 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: loading cluster ID for follower catalog manager: success
I20260619 16:54:24.274561 5774 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: acquiring CA information for follower catalog manager: success
I20260619 16:54:24.274848 5774 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 16:54:24.294003 5685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:24.294205 5685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:24.294242 5685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:24.295377 5685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:24.295439 5685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:24.295482 5685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:24.296934 5685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab
--principal=oryx/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:44817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=42151
--tserver_master_addrs=127.30.148.190:33303,127.30.148.189:35425,127.30.148.188:44279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:24.297189 5685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:24.297456 5685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:24.302276 5780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:24.302337 5782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:24.302268 5779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:24.302416 5685 server_base.cc:1034] running on GCE node
I20260619 16:54:24.302690 5685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:24.302944 5685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:24.304103 5685 hybrid_clock.cc:648] HybridClock initialized: now 1781888064304101 us; error 35 us; skew 500 ppm
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:24.306553 5685 init.cc:377] Logged in from keytab as oryx/127.30.148.129@KRBTEST.COM (short username oryx)
I20260619 16:54:24.306968 5685 webserver.cc:460] Webserver started at http://127.30.148.129:42151/ using document root <none> and password file <none>
I20260619 16:54:24.307181 5685 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:24.307261 5685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:24.308547 5685 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:24.309170 5789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.309342 5685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:24.309473 5685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "d8e6aba81fb64f3cb8e52d70d122074e"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "4931847db67ccc19609bfb1cdd4c88d5"
server_key_iv: "5365ebbb3cbb42f96292eb0adcfe9cb7"
server_key_version: "encryptionkey@0"
I20260619 16:54:24.309763 5685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:24.336762 5685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:24.339162 5685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:24.339326 5685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:24.339566 5685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:24.339890 5685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:24.339921 5685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.339941 5685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:24.339988 5685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.345445 5685 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:44817
I20260619 16:54:24.345518 5902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:44817 every 8 connection(s)
I20260619 16:54:24.345853 5685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:24.348095 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 5685
I20260619 16:54:24.348272 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 5136
I20260619 16:54:24.354266 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:38313
--local_ip_for_outbound_sockets=127.30.148.130
--tserver_master_addrs=127.30.148.190:33303
--webserver_port=46177
--webserver_interface=127.30.148.130
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab
--principal=oryx/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.148.190:33303,127.30.148.189:35425
--tserver_master_addrs=127.30.148.190:33303,127.30.148.189:35425,127.30.148.188:44279 with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.188@KRBTEST.COM
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.189@KRBTEST.COM
I20260619 16:54:24.358474 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.346842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:40151 (local address 127.30.148.190:33303)
0619 16:54:24.346887 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.346889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.347126 (+ 237us) server_negotiation.cc:407] Connection header received
0619 16:54:24.347955 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.347957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.347969 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.348002 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.349761 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.350506 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.352089 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.352387 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.352427 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.355403 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.355405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.355405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.355429 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.356827 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.357316 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.357317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.357318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.357398 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.357701 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.357702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.357703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.357846 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.357890 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.358258 (+ 368us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.358342 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:54:24.359057 5910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.346532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.129:40151)
0619 16:54:24.347064 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.347074 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.347846 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.348188 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.348188 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.349602 (+ 1414us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.349664 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.349667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.350632 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.350632 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.351986 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.351987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.352504 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.352504 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.352642 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.353686 (+ 1044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.353691 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.355302 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.356919 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.356924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.356926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.357225 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.357457 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.357457 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.357458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.357596 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.357936 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.357941 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.358090 (+ 149us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.358602 (+ 512us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.358783 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":308,"thread_start_us":109,"threads_started":1}
I20260619 16:54:24.359520 5904 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33303
I20260619 16:54:24.359609 5904 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:24.359789 5904 heartbeater.cc:511] Master 127.30.148.190:33303 requested a full tablet report, sending...
I20260619 16:54:24.360072 5908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.346531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44279 (local address 127.30.148.129:44813)
0619 16:54:24.347066 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.347074 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.347836 (+ 762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.348878 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.348879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.349139 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.349322 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.349345 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.350632 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.350632 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.351985 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.351987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.352496 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.352497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.352642 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.353564 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.353569 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.356374 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.358339 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.358341 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.358342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.358561 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.358998 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.358999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.358999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.359060 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.359731 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.359732 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.359770 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.359852 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.359958 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":403,"thread_start_us":68,"threads_started":1}
I20260619 16:54:24.360198 4940 ts_manager.cc:194] Re-registered known tserver with Master: d8e6aba81fb64f3cb8e52d70d122074e (127.30.148.129:44817)
I20260619 16:54:24.360222 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.346909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:43889 (local address 127.30.148.189:35425)
0619 16:54:24.346949 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.346951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.346962 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:54:24.347955 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.347957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.347969 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.348001 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.349974 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.350772 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.352077 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.352406 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.352446 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.357061 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.357062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.357063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.357070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.359026 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.359402 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.359403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.359403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.359458 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.359657 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.359658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.359659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.359803 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.359846 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.360032 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.360104 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:54:24.360517 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.346531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:35425 (local address 127.30.148.129:43889)
0619 16:54:24.346716 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.346757 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.347836 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.348176 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.348179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.349812 (+ 1633us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.349879 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.349881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.350868 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.350868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.351987 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.351989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.352519 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.352520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.352642 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.353457 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.353467 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.356978 (+ 3511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.359099 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.359100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.359100 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.359324 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.359531 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.359531 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.359531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.359586 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.359892 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.359893 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.359925 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.360302 (+ 377us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.360421 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":26,"thread_start_us":78,"threads_started":1}
I20260619 16:54:24.360713 4940 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:40151
I20260619 16:54:24.360906 5909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.346625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:44813 (local address 127.30.148.188:44279)
0619 16:54:24.346765 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.346767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.347135 (+ 368us) server_negotiation.cc:407] Connection header received
0619 16:54:24.347955 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.347957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.347972 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.348025 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.349461 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.350541 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.352079 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.352381 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.352439 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.356446 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.356447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.356447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.356455 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.358249 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.358844 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.358846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.358846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.358908 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.359452 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.359453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.359453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.359594 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.359656 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.360750 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.360834 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260619 16:54:24.361112 5905 heartbeater.cc:347] Connected to a master server at 127.30.148.189:35425
I20260619 16:54:24.361167 5905 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:24.361222 5903 heartbeater.cc:347] Connected to a master server at 127.30.148.188:44279
I20260619 16:54:24.361253 5905 heartbeater.cc:511] Master 127.30.148.189:35425 requested a full tablet report, sending...
I20260619 16:54:24.361261 5903 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:24.361414 5903 heartbeater.cc:511] Master 127.30.148.188:44279 requested a full tablet report, sending...
I20260619 16:54:24.361656 5538 ts_manager.cc:194] Registered new tserver with Master: d8e6aba81fb64f3cb8e52d70d122074e (127.30.148.129:44817)
I20260619 16:54:24.361759 5724 ts_manager.cc:194] Registered new tserver with Master: d8e6aba81fb64f3cb8e52d70d122074e (127.30.148.129:44817)
W20260619 16:54:24.435789 5911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:24.435961 5911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:24.435983 5911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:24.437146 5911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:24.437186 5911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:24.437235 5911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:24.438854 5911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab
--principal=oryx/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:38313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=46177
--tserver_master_addrs=127.30.148.190:33303,127.30.148.189:35425,127.30.148.188:44279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:24.439105 5911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:24.439338 5911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:54:24.444327 5911 server_base.cc:1034] running on GCE node
W20260619 16:54:24.444265 5916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:24.444278 5919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:24.444319 5917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:24.444722 5911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:24.444923 5911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:24.446074 5911 hybrid_clock.cc:648] HybridClock initialized: now 1781888064446063 us; error 28 us; skew 500 ppm
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:24.448457 5911 init.cc:377] Logged in from keytab as oryx/127.30.148.130@KRBTEST.COM (short username oryx)
I20260619 16:54:24.448868 5911 webserver.cc:460] Webserver started at http://127.30.148.130:46177/ using document root <none> and password file <none>
I20260619 16:54:24.449169 5911 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:24.449256 5911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:24.450562 5911 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:24.451138 5926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.451293 5911 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:24.451392 5911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "fb4bedcc98da432f8244bac04ed47c42"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "b0f6a0bcef16be1787b534c6ee939d54"
server_key_iv: "e24a62213c98aa28549e62c17c9e2b7e"
server_key_version: "encryptionkey@0"
I20260619 16:54:24.451722 5911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:24.482221 5911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:24.484127 5911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:24.484275 5911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:24.484500 5911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:24.484807 5911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:24.484864 5911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.484905 5911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:24.484931 5911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.490734 5911 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:38313
I20260619 16:54:24.490787 6039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:38313 every 8 connection(s)
I20260619 16:54:24.491155 5911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.188@KRBTEST.COM
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.189@KRBTEST.COM
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
I20260619 16:54:24.500102 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 5911
I20260619 16:54:24.500279 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 5270
I20260619 16:54:24.502207 5909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.491847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:40551 (local address 127.30.148.188:44279)
0619 16:54:24.491892 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.491894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.492345 (+ 451us) server_negotiation.cc:407] Connection header received
0619 16:54:24.493314 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.493315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.493372 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.493423 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.494165 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.494655 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.496115 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.496327 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.496382 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.498909 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.498910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.498910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.498917 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.500532 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.501070 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.501071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.501071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.501126 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.501430 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.501431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.501432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.501591 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.501664 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.502019 (+ 355us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.502105 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:54:24.502966 6045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.491854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44279 (local address 127.30.148.130:40551)
0619 16:54:24.492044 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.492301 (+ 257us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.493187 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.493515 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.493520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.493858 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.494064 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.494068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.494763 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.494763 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.496028 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.496030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.496409 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.496410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.496537 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.497394 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.497399 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.498802 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.500637 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.500643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.500645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.500971 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.501200 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.501201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.501202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.501339 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.501693 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.501699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.501862 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.502543 (+ 681us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.502730 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":8,"thread_start_us":109,"threads_started":1}
I20260619 16:54:24.503152 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.492154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:51839 (local address 127.30.148.189:35425)
0619 16:54:24.492196 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.492198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.492264 (+ 66us) server_negotiation.cc:407] Connection header received
0619 16:54:24.493420 (+ 1156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.493421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.493433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.493483 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.494173 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.494919 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.496427 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.496603 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.496681 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.500096 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.500098 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.500098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.500106 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.502032 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.502405 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.502406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.502406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.502456 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.502641 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.502642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.502642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.502756 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.502799 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.502985 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.503069 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:54:24.503448 6046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.491855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:35425 (local address 127.30.148.130:51839)
0619 16:54:24.492178 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.492188 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.493187 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.493515 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.493520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.493874 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.494066 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.494072 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.494999 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.494999 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.496028 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.496030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.496682 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.496683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.496763 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.497188 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.497199 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.499994 (+ 2795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.502106 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.502107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.502107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.502314 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.502521 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.502522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.502522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.502575 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.502844 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.502845 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.502880 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.503265 (+ 385us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.503353 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":49,"thread_start_us":103,"threads_started":1}
I20260619 16:54:24.503551 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.492241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:52005 (local address 127.30.148.190:33303)
0619 16:54:24.492305 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.492307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.492315 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:24.493339 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.493341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.493369 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.493425 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.494176 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.494930 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.496923 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.497108 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.497203 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.500112 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.500113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.500113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.500121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.502354 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.502728 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.502729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.502730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.502782 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.502970 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.502971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.502971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.503109 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.503151 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.503341 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.503422 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 16:54:24.503860 6040 heartbeater.cc:347] Connected to a master server at 127.30.148.188:44279
I20260619 16:54:24.503971 6040 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:24.504153 6040 heartbeater.cc:511] Master 127.30.148.188:44279 requested a full tablet report, sending...
I20260619 16:54:24.504566 5724 ts_manager.cc:194] Registered new tserver with Master: fb4bedcc98da432f8244bac04ed47c42 (127.30.148.130:38313)
I20260619 16:54:24.504618 6042 heartbeater.cc:347] Connected to a master server at 127.30.148.189:35425
I20260619 16:54:24.504638 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.491855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.130:52005)
0619 16:54:24.492146 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.492157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.493273 (+ 1116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.493526 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.493527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.493884 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.494066 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.494072 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.496129 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.496130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.496843 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.496846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.497211 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.497212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.497284 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.497708 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.497713 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.499991 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.502440 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.502440 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.502441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.502651 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.502849 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.502850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.502851 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.502906 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.503196 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.503197 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.503235 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.504386 (+ 1151us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.504494 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":19,"thread_start_us":108,"threads_started":1}
I20260619 16:54:24.504655 6042 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:24.504738 6042 heartbeater.cc:511] Master 127.30.148.189:35425 requested a full tablet report, sending...
I20260619 16:54:24.505044 5538 ts_manager.cc:194] Registered new tserver with Master: fb4bedcc98da432f8244bac04ed47c42 (127.30.148.130:38313)
I20260619 16:54:24.507144 6041 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33303
I20260619 16:54:24.507202 6041 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:24.507277 6041 heartbeater.cc:511] Master 127.30.148.190:33303 requested a full tablet report, sending...
I20260619 16:54:24.507505 4940 ts_manager.cc:194] Re-registered known tserver with Master: fb4bedcc98da432f8244bac04ed47c42 (127.30.148.130:38313)
I20260619 16:54:24.507959 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:37105
--local_ip_for_outbound_sockets=127.30.148.131
--tserver_master_addrs=127.30.148.190:33303
--webserver_port=35073
--webserver_interface=127.30.148.131
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab
--principal=oryx/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.148.190:33303,127.30.148.189:35425
--tserver_master_addrs=127.30.148.190:33303,127.30.148.189:35425,127.30.148.188:44279 with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:54:24.508004 4940 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:52005
W20260619 16:54:24.587620 6047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:24.587834 6047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:24.587873 6047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:24.589061 6047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:24.589123 6047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:24.589167 6047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:24.590626 6047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab
--principal=oryx/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:37105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=35073
--tserver_master_addrs=127.30.148.190:33303,127.30.148.189:35425,127.30.148.188:44279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:24.590867 6047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:24.591084 6047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:24.595880 6055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:24.595906 6052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:24.595942 6053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:24.596028 6047 server_base.cc:1034] running on GCE node
I20260619 16:54:24.596238 6047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:24.596462 6047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:24.597612 6047 hybrid_clock.cc:648] HybridClock initialized: now 1781888064597575 us; error 58 us; skew 500 ppm
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:24.599833 6047 init.cc:377] Logged in from keytab as oryx/127.30.148.131@KRBTEST.COM (short username oryx)
I20260619 16:54:24.600273 6047 webserver.cc:460] Webserver started at http://127.30.148.131:35073/ using document root <none> and password file <none>
I20260619 16:54:24.600481 6047 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:24.600565 6047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:24.601878 6047 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:24.602399 6062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.602569 6047 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:24.602658 6047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "9f3a049c850b48bd8bf962e2a9108377"
format_stamp: "Formatted at 2026-06-19 16:54:23 on dist-test-slave-xb9n"
server_key: "1a8f0af48552916147a8d369182bc73f"
server_key_iv: "2a0181e8fc165f420da392859328e879"
server_key_version: "encryptionkey@0"
I20260619 16:54:24.602943 6047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:24.614492 6047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:24.616158 6047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:24.616309 6047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:24.616506 6047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:24.616791 6047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:24.616844 6047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.616883 6047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:24.616910 6047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:24.622589 6047 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:37105
I20260619 16:54:24.622639 6175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:37105 every 8 connection(s)
I20260619 16:54:24.622943 6047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.189@KRBTEST.COM
Jun 19 16:54:24 dist-test-slave-xb9n krb5kdc[4895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888064, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.188@KRBTEST.COM
I20260619 16:54:24.632640 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 6047
I20260619 16:54:24.633101 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.629635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44279 (local address 127.0.0.1:58842)
0619 16:54:24.629755 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.629766 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.629799 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.629968 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.629969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.629970 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:24.630003 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.630004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.630926 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.630927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.632019 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.632022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.632865 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.632866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.632926 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.632930 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.632961 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.633007 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260619 16:54:24.633535 6183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.629657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58842 (local address 127.30.148.188:44279)
0619 16:54:24.629778 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.629780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.629809 (+ 29us) server_negotiation.cc:407] Connection header received
0619 16:54:24.629856 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.629857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.629881 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.629967 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:24.630046 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.630816 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.632112 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.632774 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.633277 (+ 503us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.633385 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.633405 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.633460 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260619 16:54:24.633777 5734 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260619 16:54:24.633965 5763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader db5925651cfe41e4840bd653b1642f06. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.634066 5763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: This master's current role is: LEARNER
I20260619 16:54:24.634189 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.623562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:42671 (local address 127.30.148.189:35425)
0619 16:54:24.623605 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.623607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.623847 (+ 240us) server_negotiation.cc:407] Connection header received
0619 16:54:24.624831 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.624832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.624845 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.624887 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.625759 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.626246 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.627282 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.627586 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.627672 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.630847 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.630849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.630849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.630857 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.632632 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.633109 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.633110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.633110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.633169 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.633475 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.633476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.633476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.633604 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.633649 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.634010 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.634085 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:54:24.634598 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.623559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:35425 (local address 127.30.148.131:42671)
0619 16:54:24.623733 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.623744 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.624727 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.624972 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.624976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.625419 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.625647 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.625651 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.626350 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.626351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.627158 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.627160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.627733 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.627734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.627869 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.628388 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.628395 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.630756 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.632732 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.632738 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.632740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.633030 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.633247 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.633248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.633248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.633399 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.633698 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.633702 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.633856 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.634296 (+ 440us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.634433 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":88,"thread_start_us":110,"threads_started":1}
I20260619 16:54:24.634967 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.623550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:51041 (local address 127.30.148.190:33303)
0619 16:54:24.623595 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.623597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.623871 (+ 274us) server_negotiation.cc:407] Connection header received
0619 16:54:24.624846 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.624848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.624860 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.624892 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.625756 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.626504 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.627686 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.627887 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.627961 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.631323 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.631324 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.631324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.631331 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.633214 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.633628 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.633629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.633629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.633687 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.633883 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.633884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.633885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.634005 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.634068 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.634788 (+ 720us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.634869 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:54:24.635162 6180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.623558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.131:51041)
0619 16:54:24.623708 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.623744 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.624727 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.624972 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.624976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.625402 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.625648 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.625654 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.626603 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.626603 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.627572 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.627574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.627971 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.627972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.628035 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.628577 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.628582 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.631255 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.633312 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.633313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.633313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.633542 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.633758 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.633759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.633759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.633819 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.634093 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.634094 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.634135 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.634207 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.634378 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":51,"thread_start_us":87,"threads_started":1}
I20260619 16:54:24.635540 6177 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33303
I20260619 16:54:24.635637 6177 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:24.635819 6177 heartbeater.cc:511] Master 127.30.148.190:33303 requested a full tablet report, sending...
I20260619 16:54:24.636185 4940 ts_manager.cc:194] Re-registered known tserver with Master: 9f3a049c850b48bd8bf962e2a9108377 (127.30.148.131:37105)
I20260619 16:54:24.636515 6178 heartbeater.cc:347] Connected to a master server at 127.30.148.189:35425
I20260619 16:54:24.636559 6178 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:24.636565 5909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.624027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:41041 (local address 127.30.148.188:44279)
0619 16:54:24.624066 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.624068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.624077 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:54:24.624846 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.624847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.624860 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.624892 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.625755 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.626506 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.627651 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.627941 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.628012 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.631814 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.631815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.631815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.631822 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.635397 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.635759 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.635760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.635760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.635813 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.635987 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.635988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.635988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.636111 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.636152 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.636370 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.636460 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:24.636696 4940 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:51041
I20260619 16:54:24.636721 6178 heartbeater.cc:511] Master 127.30.148.189:35425 requested a full tablet report, sending...
I20260619 16:54:24.636875 6182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.623844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44279 (local address 127.30.148.131:41041)
0619 16:54:24.623972 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.623982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.624731 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.624972 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.624976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.625402 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:24.625648 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.625654 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.626603 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.626603 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.627577 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.627579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.628063 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.628064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.628132 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.628754 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:24.628758 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:24.631487 (+ 2729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:24.635483 (+ 3996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.635484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.635484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.635670 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.635877 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:24.635878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:24.635878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:24.635929 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:24.636199 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:24.636200 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:24.636238 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.636625 (+ 387us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.636726 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":98,"thread_start_us":74,"threads_started":1}
I20260619 16:54:24.637053 5538 ts_manager.cc:194] Registered new tserver with Master: 9f3a049c850b48bd8bf962e2a9108377 (127.30.148.131:37105)
I20260619 16:54:24.637530 6176 heartbeater.cc:347] Connected to a master server at 127.30.148.188:44279
I20260619 16:54:24.637578 6176 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:24.637642 6176 heartbeater.cc:511] Master 127.30.148.188:44279 requested a full tablet report, sending...
I20260619 16:54:24.637930 5724 ts_manager.cc:194] Registered new tserver with Master: 9f3a049c850b48bd8bf962e2a9108377 (127.30.148.131:37105)
I20260619 16:54:24.644454 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.635031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.30.148.189:35425)
0619 16:54:24.635201 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.635203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.636417 (+ 1214us) server_negotiation.cc:407] Connection header received
0619 16:54:24.636506 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.636507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.636515 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.636562 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.637731 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.638405 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.639489 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.639766 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.639806 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.641551 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.641553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.641553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.641559 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.643179 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.643584 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.643585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.643585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.643634 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.643860 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.643861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.643861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.643982 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.644054 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.644275 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.644347 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260619 16:54:24.645731 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.636127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49718 (local address 127.30.148.190:33303)
0619 16:54:24.636155 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.636157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.638004 (+ 1847us) server_negotiation.cc:407] Connection header received
0619 16:54:24.638103 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.638104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.638113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.638141 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.638750 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.639463 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.640546 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.640827 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.640875 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.642321 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.642322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.642323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.642329 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.644260 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.644872 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.644874 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.644874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.644929 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.645139 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.645140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.645140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.645252 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.645348 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.645539 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.645622 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260619 16:54:24.646612 6183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.635775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.30.148.188:44279)
0619 16:54:24.635807 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.635809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.637823 (+ 2014us) server_negotiation.cc:407] Connection header received
0619 16:54:24.637926 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.637927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.637938 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.637993 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.638658 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.639381 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.640846 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.641476 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.641543 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.643652 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.643653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.643654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.643660 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.645405 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.645775 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.645776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.645776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.645829 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.646011 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.646012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.646012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.646135 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.646233 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.646439 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.646521 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 16:54:24.647866 4940 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:24.649369 4940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:24.650038 5688 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: NON_VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: true } } }
I20260619 16:54:24.650142 5688 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:24.652011 5688 raft_consensus.cc:1050] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06: attempting to promote NON_VOTER 8c69a169602348ffb729a47f62e547ad to VOTER
I20260619 16:54:24.652133 5688 consensus_queue.cc:237] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } }
I20260619 16:54:24.652769 5547 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 FOLLOWER]: Refusing update from remote peer db5925651cfe41e4840bd653b1642f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260619 16:54:24.652799 5733 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 1 LEARNER]: Refusing update from remote peer db5925651cfe41e4840bd653b1642f06: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260619 16:54:24.652995 6192 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 16:54:24.653079 5688 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 16:54:24.653647 5499 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 8c69a169602348ffb729a47f62e547ad (127.30.148.188) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } } }
I20260619 16:54:24.653837 5547 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 8c69a169602348ffb729a47f62e547ad (127.30.148.188) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } } }
I20260619 16:54:24.654552 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } } }
I20260619 16:54:24.654628 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:24.654686 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } } }
I20260619 16:54:24.654748 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:24.654596 5733 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 8c69a169602348ffb729a47f62e547ad (127.30.148.188) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } } }
I20260619 16:54:24.654788 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } } }
I20260619 16:54:24.654834 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:24.654970 5577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } } }
I20260619 16:54:24.655068 5577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:54:24.655702 5763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db5925651cfe41e4840bd653b1642f06" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } } }
I20260619 16:54:24.655804 5763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:54:24.659480 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.654260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:38313 (local address 127.0.0.1:45788)
0619 16:54:24.654389 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.654398 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.654437 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.656357 (+ 1920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.656358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.656359 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:24.656392 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.656393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.657193 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.657194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.658294 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.658297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.659248 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.659249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.659312 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.659315 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.659346 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.659386 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":94,"threads_started":1}
I20260619 16:54:24.659503 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.654905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49142 (local address 127.30.148.131:37105)
0619 16:54:24.656036 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.656041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.656048 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:54:24.656073 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.656075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.656086 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.656141 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:24.656328 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.657471 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.658454 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.659130 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.659231 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.659286 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.659387 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.659426 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1105,"thread_start_us":86,"threads_started":1}
I20260619 16:54:24.659682 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.654632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:44817 (local address 127.0.0.1:48510)
0619 16:54:24.655490 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.655499 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.655544 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.655871 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.655871 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.655873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:24.655921 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.655922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.657710 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.657711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.658645 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.658646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.659438 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.659439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.659502 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.659505 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.659557 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.659600 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":65,"threads_started":1}
I20260619 16:54:24.659687 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.654165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:37105 (local address 127.0.0.1:49142)
0619 16:54:24.654377 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.654388 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.654437 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.656169 (+ 1732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.656170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.656172 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:24.656225 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.656227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.657550 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.657551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.658333 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.658359 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.659232 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.659233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.659305 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.659310 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.659372 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.659412 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260619 16:54:24.659878 6203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.655087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45788 (local address 127.30.148.130:38313)
0619 16:54:24.656224 (+ 1137us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.656227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.656234 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:54:24.656261 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.656263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.656273 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.656359 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:24.656447 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.657107 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.658389 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.659130 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.659486 (+ 356us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.659639 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.659728 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.659783 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1109,"thread_start_us":76,"threads_started":1}
I20260619 16:54:24.659976 6201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.654819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48510 (local address 127.30.148.129:44817)
0619 16:54:24.655635 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.655638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.655646 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:24.655705 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.655707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.655723 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.655798 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:24.655987 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.657645 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.658730 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.659262 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.659352 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.659406 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.659857 (+ 451us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.659900 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":91,"threads_started":1}
I20260619 16:54:24.660363 6110 tablet_service.cc:1467] Processing CreateTablet for tablet 389934b44e204618b1feb94b785bd19b (DEFAULT_TABLE table=test-table [id=63c8c57a4b934e92b0915600eef81bea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:24.660468 5974 tablet_service.cc:1467] Processing CreateTablet for tablet 389934b44e204618b1feb94b785bd19b (DEFAULT_TABLE table=test-table [id=63c8c57a4b934e92b0915600eef81bea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:24.660621 6110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389934b44e204618b1feb94b785bd19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:24.660555 5837 tablet_service.cc:1467] Processing CreateTablet for tablet 389934b44e204618b1feb94b785bd19b (DEFAULT_TABLE table=test-table [id=63c8c57a4b934e92b0915600eef81bea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:24.660774 5974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389934b44e204618b1feb94b785bd19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:24.660818 5837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389934b44e204618b1feb94b785bd19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:24.663043 6204 tablet_bootstrap.cc:492] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377: Bootstrap starting.
I20260619 16:54:24.663676 6204 tablet_bootstrap.cc:654] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:24.663885 6204 log.cc:824] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:24.664037 6205 tablet_bootstrap.cc:492] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e: Bootstrap starting.
I20260619 16:54:24.664409 6206 tablet_bootstrap.cc:492] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42: Bootstrap starting.
I20260619 16:54:24.664671 6205 tablet_bootstrap.cc:654] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:24.664867 6204 tablet_bootstrap.cc:492] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377: No bootstrap required, opened a new log
I20260619 16:54:24.664948 6204 ts_tablet_manager.cc:1397] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:24.664994 6205 log.cc:824] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:24.665014 6206 tablet_bootstrap.cc:654] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:24.665321 6206 log.cc:824] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:24.666076 6206 tablet_bootstrap.cc:492] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42: No bootstrap required, opened a new log
I20260619 16:54:24.666127 6206 ts_tablet_manager.cc:1397] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:24.667122 6204 raft_consensus.cc:348] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } }
I20260619 16:54:24.667264 6204 raft_consensus.cc:374] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:24.667307 6204 raft_consensus.cc:729] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3a049c850b48bd8bf962e2a9108377, State: Initialized, Role: FOLLOWER
I20260619 16:54:24.667418 6204 consensus_queue.cc:260] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } }
I20260619 16:54:24.667671 6204 ts_tablet_manager.cc:1428] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:54:24.667752 6177 heartbeater.cc:503] Master 127.30.148.190:33303 was elected leader, sending a full tablet report...
I20260619 16:54:24.668351 6206 raft_consensus.cc:348] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } }
I20260619 16:54:24.668503 6206 raft_consensus.cc:374] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:24.668552 6206 raft_consensus.cc:729] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb4bedcc98da432f8244bac04ed47c42, State: Initialized, Role: FOLLOWER
I20260619 16:54:24.668670 6206 consensus_queue.cc:260] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } }
I20260619 16:54:24.668921 6206 ts_tablet_manager.cc:1428] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:54:24.668982 6041 heartbeater.cc:503] Master 127.30.148.190:33303 was elected leader, sending a full tablet report...
I20260619 16:54:24.669548 6205 tablet_bootstrap.cc:492] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e: No bootstrap required, opened a new log
I20260619 16:54:24.669653 6205 ts_tablet_manager.cc:1397] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260619 16:54:24.671186 6205 raft_consensus.cc:348] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } }
I20260619 16:54:24.671316 6205 raft_consensus.cc:374] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:24.671360 6205 raft_consensus.cc:729] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e6aba81fb64f3cb8e52d70d122074e, State: Initialized, Role: FOLLOWER
I20260619 16:54:24.671461 6205 consensus_queue.cc:260] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } }
I20260619 16:54:24.671695 6205 ts_tablet_manager.cc:1428] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 16:54:24.671765 5904 heartbeater.cc:503] Master 127.30.148.190:33303 was elected leader, sending a full tablet report...
W20260619 16:54:24.742451 6043 tablet.cc:2369] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:54:24.846714 5906 tablet.cc:2369] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:54:24.873617 6179 tablet.cc:2369] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:54:24.905391 6211 raft_consensus.cc:482] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:24.905510 6211 raft_consensus.cc:504] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } }
I20260619 16:54:24.905925 6211 leader_election.cc:288] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f3a049c850b48bd8bf962e2a9108377 (127.30.148.131:37105), d8e6aba81fb64f3cb8e52d70d122074e (127.30.148.129:44817)
I20260619 16:54:24.908926 6201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.906190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:47473 (local address 127.30.148.129:44817)
0619 16:54:24.906238 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.906240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.906252 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:54:24.906347 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.906348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.906360 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.906419 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:24.906615 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.907127 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.908207 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.908571 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.908627 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.908634 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.908809 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.908852 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:54:24.908963 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.906205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:45851 (local address 127.30.148.131:37105)
0619 16:54:24.906265 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.906267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.906419 (+ 152us) server_negotiation.cc:407] Connection header received
0619 16:54:24.906474 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.906475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.906484 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.906554 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:24.906698 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.907141 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.908211 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.908584 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.908675 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.908683 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.908857 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.908901 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 16:54:24.908937 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.906118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:44817 (local address 127.30.148.130:47473)
0619 16:54:24.906173 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.906186 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.906244 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.906456 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.906458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.906460 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:24.906540 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.906541 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.907203 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.907204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.908116 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.908119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.908651 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.908652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.908731 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.908737 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.908785 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.908834 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":52}
I20260619 16:54:24.909006 6045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.906140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:37105 (local address 127.30.148.130:45851)
0619 16:54:24.906311 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:24.906320 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:24.906363 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:24.906554 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:24.906555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:24.906556 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:24.906632 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.906633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.907244 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.907245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.908117 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:24.908119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.908699 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.908700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:24.908756 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.908759 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:24.908835 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 16:54:24.908870 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":153}
I20260619 16:54:24.909264 5857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389934b44e204618b1feb94b785bd19b" candidate_uuid: "fb4bedcc98da432f8244bac04ed47c42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" is_pre_election: true
I20260619 16:54:24.909449 5857 raft_consensus.cc:2454] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb4bedcc98da432f8244bac04ed47c42 in term 0.
I20260619 16:54:24.909451 6130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389934b44e204618b1feb94b785bd19b" candidate_uuid: "fb4bedcc98da432f8244bac04ed47c42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3a049c850b48bd8bf962e2a9108377" is_pre_election: true
I20260619 16:54:24.909610 6130 raft_consensus.cc:2454] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb4bedcc98da432f8244bac04ed47c42 in term 0.
I20260619 16:54:24.909674 5929 leader_election.cc:302] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8e6aba81fb64f3cb8e52d70d122074e, fb4bedcc98da432f8244bac04ed47c42; no voters:
I20260619 16:54:24.909788 6211 raft_consensus.cc:2781] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:24.909840 6211 raft_consensus.cc:482] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:24.909869 6211 raft_consensus.cc:3037] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:24.910652 6211 raft_consensus.cc:504] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } }
I20260619 16:54:24.910786 6211 leader_election.cc:288] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [CANDIDATE]: Term 1 election: Requested vote from peers 9f3a049c850b48bd8bf962e2a9108377 (127.30.148.131:37105), d8e6aba81fb64f3cb8e52d70d122074e (127.30.148.129:44817)
I20260619 16:54:24.910912 6130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389934b44e204618b1feb94b785bd19b" candidate_uuid: "fb4bedcc98da432f8244bac04ed47c42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3a049c850b48bd8bf962e2a9108377"
I20260619 16:54:24.910943 5857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389934b44e204618b1feb94b785bd19b" candidate_uuid: "fb4bedcc98da432f8244bac04ed47c42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e6aba81fb64f3cb8e52d70d122074e"
I20260619 16:54:24.910991 6130 raft_consensus.cc:3037] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:24.911002 5857 raft_consensus.cc:3037] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:24.911818 5857 raft_consensus.cc:2454] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb4bedcc98da432f8244bac04ed47c42 in term 1.
I20260619 16:54:24.911818 6130 raft_consensus.cc:2454] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb4bedcc98da432f8244bac04ed47c42 in term 1.
I20260619 16:54:24.911983 5927 leader_election.cc:302] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f3a049c850b48bd8bf962e2a9108377, fb4bedcc98da432f8244bac04ed47c42; no voters:
I20260619 16:54:24.912083 6211 raft_consensus.cc:2781] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:24.912261 6211 raft_consensus.cc:686] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 1 LEADER]: Becoming Leader. State: Replica: fb4bedcc98da432f8244bac04ed47c42, State: Running, Role: LEADER
I20260619 16:54:24.912482 6211 consensus_queue.cc:237] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } }
I20260619 16:54:24.913414 4940 catalog_manager.cc:5495] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb4bedcc98da432f8244bac04ed47c42 (127.30.148.130). New cstate: current_term: 1 leader_uuid: "fb4bedcc98da432f8244bac04ed47c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } health_report { overall_health: HEALTHY } } }
I20260619 16:54:24.939278 6203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.936727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45796 (local address 127.30.148.130:38313)
0619 16:54:24.936804 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.936806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.936814 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:24.936869 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.936870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.936890 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.936961 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:24.937114 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.937629 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.938374 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.938664 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.938738 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.938933 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:24.939029 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:24.939147 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.939172 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.939189 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 16:54:24.940914 5857 raft_consensus.cc:1261] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e [term 1 FOLLOWER]: Refusing update from remote peer fb4bedcc98da432f8244bac04ed47c42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:24.940941 6130 raft_consensus.cc:1261] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377 [term 1 FOLLOWER]: Refusing update from remote peer fb4bedcc98da432f8244bac04ed47c42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:24.941239 6211 consensus_queue.cc:1035] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:24.941375 6213 consensus_queue.cc:1035] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:24.944258 6218 mvcc.cc:205] Tried to move back new op lower bound from 7298613513995603968 to 7298613513882046464. Current Snapshot: MvccSnapshot[applied={T|T < 7298613513995603968}]
I20260619 16:54:24.944470 6216 mvcc.cc:205] Tried to move back new op lower bound from 7298613513995603968 to 7298613513882046464. Current Snapshot: MvccSnapshot[applied={T|T < 7298613513995603968}]
I20260619 16:54:24.944938 6217 mvcc.cc:205] Tried to move back new op lower bound from 7298613513995603968 to 7298613513882046464. Current Snapshot: MvccSnapshot[applied={T|T < 7298613513995603968}]
I20260619 16:54:24.947393 4939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:24.947480 4939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:24.948925 4939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db5925651cfe41e4840bd653b1642f06: Sending DeleteTablet for 3 replicas of tablet 389934b44e204618b1feb94b785bd19b
I20260619 16:54:24.949301 5974 tablet_service.cc:1514] Processing DeleteTablet for tablet 389934b44e204618b1feb94b785bd19b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:24 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:45788
I20260619 16:54:24.949301 6110 tablet_service.cc:1514] Processing DeleteTablet for tablet 389934b44e204618b1feb94b785bd19b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:24 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:49142
I20260619 16:54:24.949546 6227 tablet_replica.cc:330] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377: stopping tablet replica
I20260619 16:54:24.949664 6227 raft_consensus.cc:2229] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:24.949780 6227 raft_consensus.cc:2258] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:24.949862 5837 tablet_service.cc:1514] Processing DeleteTablet for tablet 389934b44e204618b1feb94b785bd19b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:24 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:48510
I20260619 16:54:24.950191 6226 tablet_replica.cc:330] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42: stopping tablet replica
I20260619 16:54:24.950287 6226 raft_consensus.cc:2229] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:24.950302 6227 ts_tablet_manager.cc:1905] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:24.950392 6226 raft_consensus.cc:2258] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:24.950775 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 4908
I20260619 16:54:24.950798 6226 ts_tablet_manager.cc:1905] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:24.951943 6227 ts_tablet_manager.cc:1918] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:24.952061 6227 log.cc:1196] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/389934b44e204618b1feb94b785bd19b
I20260619 16:54:24.952364 6227 ts_tablet_manager.cc:1939] T 389934b44e204618b1feb94b785bd19b P 9f3a049c850b48bd8bf962e2a9108377: Deleting consensus metadata
I20260619 16:54:24.952863 6226 ts_tablet_manager.cc:1918] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:24.952935 6226 log.cc:1196] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/389934b44e204618b1feb94b785bd19b
I20260619 16:54:24.953207 6226 ts_tablet_manager.cc:1939] T 389934b44e204618b1feb94b785bd19b P fb4bedcc98da432f8244bac04ed47c42: Deleting consensus metadata
I20260619 16:54:24.953444 6229 tablet_replica.cc:330] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e: stopping tablet replica
I20260619 16:54:24.953570 6229 raft_consensus.cc:2229] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:24.953702 6229 raft_consensus.cc:2258] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:24.954203 6229 ts_tablet_manager.cc:1905] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:24.956097 6229 ts_tablet_manager.cc:1918] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:24.956171 6229 log.cc:1196] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-0/wal/wals/389934b44e204618b1feb94b785bd19b
I20260619 16:54:24.956424 6229 ts_tablet_manager.cc:1939] T 389934b44e204618b1feb94b785bd19b P d8e6aba81fb64f3cb8e52d70d122074e: Deleting consensus metadata
W20260619 16:54:24.957819 5930 connection.cc:535] server connection from 127.0.0.1:45788 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 16:54:24.957839 6064 connection.cc:535] server connection from 127.0.0.1:49142 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 16:54:24.958089 5791 connection.cc:535] server connection from 127.0.0.1:48510 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 16:54:24.964814 6183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.949814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58858 (local address 127.30.148.188:44279)
0619 16:54:24.950017 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.950019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.953938 (+ 3919us) server_negotiation.cc:407] Connection header received
0619 16:54:24.954068 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.954069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.954079 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.954108 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.954972 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.955798 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.957065 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.957418 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.957463 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.959262 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.959263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.959263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.959270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.961119 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.963988 (+ 2869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.963990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.963990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.964059 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.964273 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.964273 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.964274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.964384 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.964452 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.964655 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.964730 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260619 16:54:24.968077 5909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.959838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58866 (local address 127.30.148.188:44279)
0619 16:54:24.959904 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.959906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.959914 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:24.959936 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.959936 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.959946 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.960008 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.960928 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.961732 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.962680 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.962967 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.963035 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.964478 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.964479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.964479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.964486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.966440 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.967154 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.967155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.967156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.967221 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.967442 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.967443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.967443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.967579 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.967673 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.967944 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.968011 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 16:54:24.970362 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:24.960396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35700 (local address 127.30.148.189:35425)
0619 16:54:24.960534 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:24.960536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:24.961339 (+ 803us) server_negotiation.cc:407] Connection header received
0619 16:54:24.963404 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:24.963405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:24.963416 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:24.963476 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:24.964245 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.964906 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.966075 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:24.966265 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:24.966347 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:24.967748 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:24.967749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:24.967750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:24.967756 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:24.969256 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.969632 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.969633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.969634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.969686 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:24.969893 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:24.969894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:24.969894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:24.970000 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:24.970094 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:24.970203 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 16:54:24.970270 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260619 16:54:25.945772 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:25.945547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.130:57779)
0619 16:54:25.945672 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:25.945738 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":53,"threads_started":1}
I20260619 16:54:25.945772 6236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:25.945476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.129:57713)
0619 16:54:25.945640 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:25.945717 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":59,"threads_started":1}
I20260619 16:54:25.945873 6238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:25.945612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.131:32821)
0619 16:54:25.945737 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:25.945802 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":41,"threads_started":1}
W20260619 16:54:25.945991 5904 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:33303 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
W20260619 16:54:25.946064 6177 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:33303 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
W20260619 16:54:25.946249 6041 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:33303 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
I20260619 16:54:26.449353 6239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader db5925651cfe41e4840bd653b1642f06)
I20260619 16:54:26.449529 6239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } }
I20260619 16:54:26.449751 6240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 1 FOLLOWER]: Starting pre-election (detected failure of leader db5925651cfe41e4840bd653b1642f06)
I20260619 16:54:26.449817 6239 leader_election.cc:288] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303), 8c69a169602348ffb729a47f62e547ad (127.30.148.188:44279)
I20260619 16:54:26.449935 6240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } }
I20260619 16:54:26.450202 6241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.449904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50202)
0619 16:54:26.449997 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.450043 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":40,"threads_started":1}
I20260619 16:54:26.450305 6240 leader_election.cc:288] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303), 256dd41ec994418b9fd88957eeb58a5a (127.30.148.189:35425)
W20260619 16:54:26.450313 5522 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
I20260619 16:54:26.450670 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.450367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50204)
0619 16:54:26.450583 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.450628 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":70,"threads_started":1}
I20260619 16:54:26.450785 6241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.450684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50210)
0619 16:54:26.450733 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.450759 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 16:54:26.450786 5708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
W20260619 16:54:26.450861 5522 leader_election.cc:334] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303): Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
I20260619 16:54:26.451256 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.451101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50220)
0619 16:54:26.451207 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.451230 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
W20260619 16:54:26.451324 5708 leader_election.cc:334] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303): Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
I20260619 16:54:26.453197 6242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.449965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44279 (local address 127.0.0.1:58870)
0619 16:54:26.450139 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.450150 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:26.450205 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:26.450589 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:26.450590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:26.450592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:26.450649 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:26.450651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:26.451638 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:26.451639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:26.452429 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:26.452431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:26.452991 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:26.452992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:26.453049 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:26.453055 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 16:54:26.453109 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 16:54:26.453152 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":50,"threads_started":1}
I20260619 16:54:26.453378 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.450404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:35425 (local address 127.0.0.1:35708)
0619 16:54:26.450671 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.450681 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:26.450736 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:26.450924 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:26.450925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:26.450927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:26.450969 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:26.450971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:26.451807 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:26.451807 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:26.452564 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:26.452566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:26.453119 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:26.453120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:26.453174 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:26.453178 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:26.453242 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 16:54:26.453286 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":36,"threads_started":1}
I20260619 16:54:26.453387 6247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.450518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35708 (local address 127.30.148.189:35425)
0619 16:54:26.450639 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:26.450640 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:26.450787 (+ 147us) server_negotiation.cc:407] Connection header received
0619 16:54:26.450810 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:26.450811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:26.450821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:26.450878 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:26.451054 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:26.451732 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:26.452642 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:26.453012 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:26.453090 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:26.453099 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:26.453255 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 16:54:26.453296 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":83,"threads_started":1}
I20260619 16:54:26.453229 6243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.450083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58870 (local address 127.30.148.188:44279)
0619 16:54:26.450413 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:26.450415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:26.450425 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:26.450450 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:26.450451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:26.450463 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:26.450531 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:26.450775 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:26.451558 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:26.452513 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:26.452920 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:26.452975 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:26.452982 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:26.453121 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 16:54:26.453158 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":68,"threads_started":1}
I20260619 16:54:26.453580 5547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c69a169602348ffb729a47f62e547ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "256dd41ec994418b9fd88957eeb58a5a" is_pre_election: true
I20260619 16:54:26.453595 5733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "256dd41ec994418b9fd88957eeb58a5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8c69a169602348ffb729a47f62e547ad" is_pre_election: true
I20260619 16:54:26.453677 5733 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 256dd41ec994418b9fd88957eeb58a5a in term 1.
I20260619 16:54:26.453677 5547 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c69a169602348ffb729a47f62e547ad in term 1.
I20260619 16:54:26.453819 5524 leader_election.cc:302] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 256dd41ec994418b9fd88957eeb58a5a, 8c69a169602348ffb729a47f62e547ad; no voters: db5925651cfe41e4840bd653b1642f06
I20260619 16:54:26.453825 5711 leader_election.cc:302] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 256dd41ec994418b9fd88957eeb58a5a, 8c69a169602348ffb729a47f62e547ad; no voters: db5925651cfe41e4840bd653b1642f06
I20260619 16:54:26.453925 6239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 16:54:26.453935 6240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 16:54:26.453963 6239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader db5925651cfe41e4840bd653b1642f06)
I20260619 16:54:26.453985 6239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 1 FOLLOWER]: Advancing to term 2
I20260619 16:54:26.453981 6240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 1 FOLLOWER]: Starting leader election (detected failure of leader db5925651cfe41e4840bd653b1642f06)
I20260619 16:54:26.454013 6240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 1 FOLLOWER]: Advancing to term 2
I20260619 16:54:26.454739 6239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } }
I20260619 16:54:26.454773 6240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } }
I20260619 16:54:26.454866 6239 leader_election.cc:288] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [CANDIDATE]: Term 2 election: Requested vote from peers db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303), 8c69a169602348ffb729a47f62e547ad (127.30.148.188:44279)
I20260619 16:54:26.454901 6240 leader_election.cc:288] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [CANDIDATE]: Term 2 election: Requested vote from peers db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303), 256dd41ec994418b9fd88957eeb58a5a (127.30.148.189:35425)
I20260619 16:54:26.455025 6242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.454936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50230)
0619 16:54:26.454970 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.454998 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 16:54:26.455004 5733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "256dd41ec994418b9fd88957eeb58a5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8c69a169602348ffb729a47f62e547ad"
I20260619 16:54:26.455089 5733 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 256dd41ec994418b9fd88957eeb58a5a in current term 2: Already voted for candidate 8c69a169602348ffb729a47f62e547ad in this term.
I20260619 16:54:26.455134 5547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c69a169602348ffb729a47f62e547ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "256dd41ec994418b9fd88957eeb58a5a"
I20260619 16:54:26.455225 5547 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c69a169602348ffb729a47f62e547ad in current term 2: Already voted for candidate 256dd41ec994418b9fd88957eeb58a5a in this term.
I20260619 16:54:26.455312 6242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.455207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50248)
0619 16:54:26.455244 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.455263 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 16:54:26.455358 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.455009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50238)
0619 16:54:26.455303 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.455332 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281}
W20260619 16:54:26.455363 5522 leader_election.cc:334] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303): Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
I20260619 16:54:26.455389 5522 leader_election.cc:302] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 256dd41ec994418b9fd88957eeb58a5a; no voters: 8c69a169602348ffb729a47f62e547ad, db5925651cfe41e4840bd653b1642f06
I20260619 16:54:26.455506 6239 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 16:54:26.455623 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.455550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50252)
0619 16:54:26.455584 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.455604 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260619 16:54:26.455721 5708 leader_election.cc:334] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303): Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
I20260619 16:54:26.455767 5708 leader_election.cc:302] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c69a169602348ffb729a47f62e547ad; no voters: 256dd41ec994418b9fd88957eeb58a5a, db5925651cfe41e4840bd653b1642f06
I20260619 16:54:26.455845 6240 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 16:54:26.946524 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.946276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.129:47287)
0619 16:54:26.946420 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.946480 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260619 16:54:26.946553 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.946327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.131:50931)
0619 16:54:26.946465 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.946500 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260619 16:54:26.946702 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:26.946503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.130:38443)
0619 16:54:26.946588 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:26.946658 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":38,"threads_started":1}
I20260619 16:54:27.947194 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:27.946945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.131:39171)
0619 16:54:27.947113 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:27.947155 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":72,"threads_started":1}
I20260619 16:54:27.947197 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:27.946968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.129:36861)
0619 16:54:27.947099 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:27.947141 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260619 16:54:27.947320 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:27.947124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.130:34881)
0619 16:54:27.947209 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:27.947238 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":38,"threads_started":1}
I20260619 16:54:28.070135 6256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:28.070258 6256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } }
I20260619 16:54:28.070472 6256 leader_election.cc:288] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303), 256dd41ec994418b9fd88957eeb58a5a (127.30.148.189:35425)
I20260619 16:54:28.070716 5547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c69a169602348ffb729a47f62e547ad" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "256dd41ec994418b9fd88957eeb58a5a" is_pre_election: true
I20260619 16:54:28.070811 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.070619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50486)
0619 16:54:28.070720 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.070766 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":54,"threads_started":1}
I20260619 16:54:28.070828 5547 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c69a169602348ffb729a47f62e547ad in term 2.
I20260619 16:54:28.070992 5711 leader_election.cc:302] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 256dd41ec994418b9fd88957eeb58a5a, 8c69a169602348ffb729a47f62e547ad; no voters:
I20260619 16:54:28.071100 6256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 16:54:28.071144 6256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:28.071159 6256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 2 FOLLOWER]: Advancing to term 3
I20260619 16:54:28.071270 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.071169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50496)
0619 16:54:28.071212 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.071236 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260619 16:54:28.071376 5708 leader_election.cc:334] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303): Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
I20260619 16:54:28.072014 6256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 3 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } }
I20260619 16:54:28.072149 6256 leader_election.cc:288] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [CANDIDATE]: Term 3 election: Requested vote from peers db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303), 256dd41ec994418b9fd88957eeb58a5a (127.30.148.189:35425)
I20260619 16:54:28.072304 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.072220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50498)
0619 16:54:28.072260 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.072282 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 16:54:28.072335 5547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c69a169602348ffb729a47f62e547ad" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "256dd41ec994418b9fd88957eeb58a5a"
I20260619 16:54:28.072450 5547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 2 FOLLOWER]: Advancing to term 3
I20260619 16:54:28.072559 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.072494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50508)
0619 16:54:28.072523 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.072541 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260619 16:54:28.072613 5708 leader_election.cc:334] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303): Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
I20260619 16:54:28.073189 5547 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c69a169602348ffb729a47f62e547ad in term 3.
I20260619 16:54:28.073340 5711 leader_election.cc:302] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 256dd41ec994418b9fd88957eeb58a5a, 8c69a169602348ffb729a47f62e547ad; no voters: db5925651cfe41e4840bd653b1642f06
I20260619 16:54:28.073458 6256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 3 FOLLOWER]: Leader election won for term 3
I20260619 16:54:28.073578 6256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [term 3 LEADER]: Becoming Leader. State: Replica: 8c69a169602348ffb729a47f62e547ad, State: Running, Role: LEADER
I20260619 16:54:28.073660 6256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } }
I20260619 16:54:28.074020 6259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c69a169602348ffb729a47f62e547ad. Latest consensus state: current_term: 3 leader_uuid: "8c69a169602348ffb729a47f62e547ad" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } } }
I20260619 16:54:28.074116 6259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:28.074285 6261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:28.074725 6261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:28.074777 6261 catalog_manager.cc:1259] Loaded cluster ID: 6ebe8142fd404b3380dfaf6332afbf45
I20260619 16:54:28.074793 6261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:28.074932 6261 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:28.074981 6261 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: Loaded TSK: 0
I20260619 16:54:28.075088 6261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:54:28.111788 5724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:28.113322 5724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:28.114358 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.114230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50520)
0619 16:54:28.114307 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.114335 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 16:54:28.114406 5547 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [term 3 FOLLOWER]: Refusing update from remote peer 8c69a169602348ffb729a47f62e547ad: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260619 16:54:28.114599 6259 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 16:54:28.114677 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.114565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.0.0.1:50522)
0619 16:54:28.114630 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.114656 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 16:54:28.114773 5708 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad -> Peer db5925651cfe41e4840bd653b1642f06 (127.30.148.190:33303): Couldn't send request to peer db5925651cfe41e4840bd653b1642f06. Status: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 16:54:28.115231 6263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c69a169602348ffb729a47f62e547ad. Latest consensus state: current_term: 3 leader_uuid: "8c69a169602348ffb729a47f62e547ad" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } } }
I20260619 16:54:28.115338 6263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:54:28.115675 6262 mvcc.cc:205] Tried to move back new op lower bound from 7298613526994411520 to 7298613526830350336. Current Snapshot: MvccSnapshot[applied={T|T < 7298613526994411520}]
I20260619 16:54:28.115782 6256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "8c69a169602348ffb729a47f62e547ad" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } } }
I20260619 16:54:28.115864 6256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:28.116081 6263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "8c69a169602348ffb729a47f62e547ad" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "db5925651cfe41e4840bd653b1642f06" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33303 } } peers { permanent_uuid: "256dd41ec994418b9fd88957eeb58a5a" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 35425 } attrs { promote: false } } peers { permanent_uuid: "8c69a169602348ffb729a47f62e547ad" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44279 } attrs { promote: false } } }
I20260619 16:54:28.116159 6263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 256dd41ec994418b9fd88957eeb58a5a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:54:28.116119 6264 mvcc.cc:205] Tried to move back new op lower bound from 7298613526994411520 to 7298613526830350336. Current Snapshot: MvccSnapshot[applied={T|T < 7298613526994411520}]
I20260619 16:54:28.120800 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.117565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:38313 (local address 127.0.0.1:45808)
0619 16:54:28.118042 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.118054 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:28.118102 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:28.118302 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:28.118303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:28.118305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:28.118343 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:28.118344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.119168 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.119169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:28.120015 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:28.120017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.120637 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.120638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:28.120679 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:28.120682 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:28.120726 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 16:54:28.120754 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":1,"thread_start_us":75,"threads_started":1}
I20260619 16:54:28.120846 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.117668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45808 (local address 127.30.148.130:38313)
0619 16:54:28.117760 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:28.117761 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:28.118148 (+ 387us) server_negotiation.cc:407] Connection header received
0619 16:54:28.118176 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:28.118177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:28.118187 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:28.118252 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:28.118414 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.119109 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.120096 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.120564 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.120636 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:28.120643 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:28.120743 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 16:54:28.120776 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":49,"threads_started":1}
I20260619 16:54:28.121169 5974 tablet_service.cc:1467] Processing CreateTablet for tablet ba2d63183fde431daa603909d61350b8 (DEFAULT_TABLE table=test-table [id=469f910c92f94639aa906b5896230470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:28.121294 6272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.117664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48512 (local address 127.30.148.129:44817)
0619 16:54:28.117962 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:28.117964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:28.118232 (+ 268us) server_negotiation.cc:407] Connection header received
0619 16:54:28.118316 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:28.118316 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:28.118326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:28.118386 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:28.118483 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.118955 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.120386 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.120925 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.120996 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:28.121003 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:28.121160 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 16:54:28.121219 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":51,"threads_started":1}
I20260619 16:54:28.121318 5974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba2d63183fde431daa603909d61350b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:28.121534 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.117584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:44817 (local address 127.0.0.1:48512)
0619 16:54:28.118174 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.118183 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:28.118254 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:28.118388 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:28.118388 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:28.118390 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:28.118433 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:28.118434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.119039 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.119040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:28.120300 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:28.120302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.121009 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.121010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:28.121077 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:28.121080 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:28.121427 (+ 347us) client_negotiation.cc:240] Negotiation successful
0619 16:54:28.121470 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":45,"thread_start_us":81,"threads_started":1}
I20260619 16:54:28.121770 5837 tablet_service.cc:1467] Processing CreateTablet for tablet ba2d63183fde431daa603909d61350b8 (DEFAULT_TABLE table=test-table [id=469f910c92f94639aa906b5896230470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:28.121887 5837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba2d63183fde431daa603909d61350b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:28.122810 6273 tablet_bootstrap.cc:492] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42: Bootstrap starting.
I20260619 16:54:28.123179 6273 tablet_bootstrap.cc:654] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:28.123464 6274 tablet_bootstrap.cc:492] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e: Bootstrap starting.
I20260619 16:54:28.123670 6257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.117474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:37105 (local address 127.0.0.1:49156)
0619 16:54:28.117614 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.117731 (+ 117us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:28.117841 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:28.118113 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:28.118114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:28.118116 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:28.118169 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:28.118171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.119114 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.119114 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:28.120300 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:28.120302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.123421 (+ 3119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.123422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:28.123485 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:28.123488 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:28.123571 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 16:54:28.123610 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":36}
I20260619 16:54:28.123739 6273 tablet_bootstrap.cc:492] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42: No bootstrap required, opened a new log
I20260619 16:54:28.123786 6273 ts_tablet_manager.cc:1397] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:28.123906 6268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.117538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49156 (local address 127.30.148.131:37105)
0619 16:54:28.117696 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:28.117699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:28.117785 (+ 86us) server_negotiation.cc:407] Connection header received
0619 16:54:28.117949 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:28.117950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:28.117963 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:28.118061 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:28.118229 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.119007 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.121038 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.123313 (+ 2275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.123412 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:28.123420 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:28.123777 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 16:54:28.123825 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":48,"threads_started":1}
I20260619 16:54:28.123906 6273 raft_consensus.cc:348] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } }
I20260619 16:54:28.123968 6273 raft_consensus.cc:374] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:28.123991 6273 raft_consensus.cc:729] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb4bedcc98da432f8244bac04ed47c42, State: Initialized, Role: FOLLOWER
I20260619 16:54:28.124048 6273 consensus_queue.cc:260] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } }
I20260619 16:54:28.124099 6110 tablet_service.cc:1467] Processing CreateTablet for tablet ba2d63183fde431daa603909d61350b8 (DEFAULT_TABLE table=test-table [id=469f910c92f94639aa906b5896230470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:28.124195 6110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba2d63183fde431daa603909d61350b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:28.124226 6273 ts_tablet_manager.cc:1428] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:28.124297 6274 tablet_bootstrap.cc:654] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:28.124562 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.124461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.130:53063)
0619 16:54:28.124507 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.124531 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 16:54:28.124893 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.124815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.130:49753)
0619 16:54:28.124848 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.124869 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 16:54:28.124902 6274 tablet_bootstrap.cc:492] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e: No bootstrap required, opened a new log
I20260619 16:54:28.124944 6274 ts_tablet_manager.cc:1397] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:28.125108 6040 heartbeater.cc:503] Master 127.30.148.188:44279 was elected leader, sending a full tablet report...
I20260619 16:54:28.125085 6274 raft_consensus.cc:348] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } }
I20260619 16:54:28.125144 6274 raft_consensus.cc:374] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:28.125164 6274 raft_consensus.cc:729] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e6aba81fb64f3cb8e52d70d122074e, State: Initialized, Role: FOLLOWER
I20260619 16:54:28.125219 6274 consensus_queue.cc:260] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } }
I20260619 16:54:28.125440 6274 ts_tablet_manager.cc:1428] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:28.125730 6279 tablet_bootstrap.cc:492] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377: Bootstrap starting.
I20260619 16:54:28.125844 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.125690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.129:55655)
0619 16:54:28.125789 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.125817 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 16:54:28.126022 6279 tablet_bootstrap.cc:654] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:28.126458 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.126373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.129:47461)
0619 16:54:28.126407 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.126435 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 16:54:28.126619 5903 heartbeater.cc:503] Master 127.30.148.188:44279 was elected leader, sending a full tablet report...
I20260619 16:54:28.126657 6276 raft_consensus.cc:482] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:28.126724 6276 raft_consensus.cc:504] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } }
I20260619 16:54:28.126850 6276 leader_election.cc:288] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f3a049c850b48bd8bf962e2a9108377 (127.30.148.131:37105), d8e6aba81fb64f3cb8e52d70d122074e (127.30.148.129:44817)
I20260619 16:54:28.126960 6130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba2d63183fde431daa603909d61350b8" candidate_uuid: "fb4bedcc98da432f8244bac04ed47c42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3a049c850b48bd8bf962e2a9108377" is_pre_election: true
I20260619 16:54:28.126988 5857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba2d63183fde431daa603909d61350b8" candidate_uuid: "fb4bedcc98da432f8244bac04ed47c42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" is_pre_election: true
I20260619 16:54:28.127048 5857 raft_consensus.cc:2454] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb4bedcc98da432f8244bac04ed47c42 in term 0.
I20260619 16:54:28.127198 5929 leader_election.cc:302] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8e6aba81fb64f3cb8e52d70d122074e, fb4bedcc98da432f8244bac04ed47c42; no voters:
I20260619 16:54:28.127233 6279 tablet_bootstrap.cc:492] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377: No bootstrap required, opened a new log
I20260619 16:54:28.127290 6276 raft_consensus.cc:2781] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260619 16:54:28.127287 5927 leader_election.cc:341] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f3a049c850b48bd8bf962e2a9108377 (127.30.148.131:37105): Illegal state: must be running to vote when last-logged opid is not known
I20260619 16:54:28.127374 6276 raft_consensus.cc:482] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:28.127396 6276 raft_consensus.cc:3037] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:28.127962 6276 raft_consensus.cc:504] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } }
I20260619 16:54:28.128093 6276 leader_election.cc:288] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [CANDIDATE]: Term 1 election: Requested vote from peers 9f3a049c850b48bd8bf962e2a9108377 (127.30.148.131:37105), d8e6aba81fb64f3cb8e52d70d122074e (127.30.148.129:44817)
I20260619 16:54:28.127300 6279 ts_tablet_manager.cc:1397] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:28.128204 6130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba2d63183fde431daa603909d61350b8" candidate_uuid: "fb4bedcc98da432f8244bac04ed47c42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3a049c850b48bd8bf962e2a9108377"
I20260619 16:54:28.128217 5857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba2d63183fde431daa603909d61350b8" candidate_uuid: "fb4bedcc98da432f8244bac04ed47c42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e6aba81fb64f3cb8e52d70d122074e"
I20260619 16:54:28.128275 5857 raft_consensus.cc:3037] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:28.128259 6279 raft_consensus.cc:348] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } }
I20260619 16:54:28.128334 6279 raft_consensus.cc:374] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:28.128368 6279 raft_consensus.cc:729] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3a049c850b48bd8bf962e2a9108377, State: Initialized, Role: FOLLOWER
I20260619 16:54:28.128434 6279 consensus_queue.cc:260] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } }
I20260619 16:54:28.128518 6130 raft_consensus.cc:3037] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:28.128620 6279 ts_tablet_manager.cc:1428] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:28.128965 5857 raft_consensus.cc:2454] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb4bedcc98da432f8244bac04ed47c42 in term 1.
I20260619 16:54:28.129045 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.128932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.131:47439)
0619 16:54:28.128988 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.129018 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 16:54:28.129104 5929 leader_election.cc:302] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8e6aba81fb64f3cb8e52d70d122074e, fb4bedcc98da432f8244bac04ed47c42; no voters:
I20260619 16:54:28.129172 6276 raft_consensus.cc:2781] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:28.129194 6130 raft_consensus.cc:2454] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb4bedcc98da432f8244bac04ed47c42 in term 1.
I20260619 16:54:28.129281 6276 raft_consensus.cc:686] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 1 LEADER]: Becoming Leader. State: Replica: fb4bedcc98da432f8244bac04ed47c42, State: Running, Role: LEADER
I20260619 16:54:28.129334 6276 consensus_queue.cc:237] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } }
I20260619 16:54:28.129657 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.129444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.130:52871)
0619 16:54:28.129599 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.129629 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260619 16:54:28.129871 6176 heartbeater.cc:503] Master 127.30.148.188:44279 was elected leader, sending a full tablet report...
I20260619 16:54:28.130198 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.130104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.131:53599)
0619 16:54:28.130136 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.130166 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260619 16:54:28.130373 5724 catalog_manager.cc:5495] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb4bedcc98da432f8244bac04ed47c42 (127.30.148.130). New cstate: current_term: 1 leader_uuid: "fb4bedcc98da432f8244bac04ed47c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb4bedcc98da432f8244bac04ed47c42" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 } health_report { overall_health: UNKNOWN } } }
I20260619 16:54:28.137791 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.135644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45816 (local address 127.30.148.130:38313)
0619 16:54:28.135677 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:28.135679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:28.135688 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:54:28.135709 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:28.135709 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:28.135717 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:28.135771 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:28.135920 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.136358 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.137095 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:28.137298 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:28.137392 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:28.137547 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:28.137578 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:28.137647 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:28.137707 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 16:54:28.137726 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 16:54:28.138465 5857 raft_consensus.cc:1261] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e [term 1 FOLLOWER]: Refusing update from remote peer fb4bedcc98da432f8244bac04ed47c42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:28.138469 6130 raft_consensus.cc:1261] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377 [term 1 FOLLOWER]: Refusing update from remote peer fb4bedcc98da432f8244bac04ed47c42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:28.138679 6282 consensus_queue.cc:1035] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3a049c850b48bd8bf962e2a9108377" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:28.138768 6276 consensus_queue.cc:1035] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8e6aba81fb64f3cb8e52d70d122074e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 44817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:28.139858 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.139760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.131:37415)
0619 16:54:28.139793 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.139827 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 16:54:28.139865 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.139475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.129:51905)
0619 16:54:28.139806 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.139834 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316}
I20260619 16:54:28.141156 6255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.141032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.130:40585)
0619 16:54:28.141104 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.141133 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 16:54:28.141330 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.141259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.129:37259)
0619 16:54:28.141291 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.141309 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 16:54:28.141330 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:28.141243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33303 (local address 127.30.148.131:39697)
0619 16:54:28.141281 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.141305 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 16:54:28.141551 5723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:28.141641 5723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:28.142774 5723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c69a169602348ffb729a47f62e547ad: Sending DeleteTablet for 3 replicas of tablet ba2d63183fde431daa603909d61350b8
I20260619 16:54:28.143060 5837 tablet_service.cc:1514] Processing DeleteTablet for tablet ba2d63183fde431daa603909d61350b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:28 UTC) from {username='oryx', principal='oryx/127.30.148.188@KRBTEST.COM'} at 127.0.0.1:48512
I20260619 16:54:28.143095 6110 tablet_service.cc:1514] Processing DeleteTablet for tablet ba2d63183fde431daa603909d61350b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:28 UTC) from {username='oryx', principal='oryx/127.30.148.188@KRBTEST.COM'} at 127.0.0.1:49156
I20260619 16:54:28.143110 5974 tablet_service.cc:1514] Processing DeleteTablet for tablet ba2d63183fde431daa603909d61350b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:28 UTC) from {username='oryx', principal='oryx/127.30.148.188@KRBTEST.COM'} at 127.0.0.1:45808
I20260619 16:54:28.143234 6295 tablet_replica.cc:330] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e: stopping tablet replica
I20260619 16:54:28.143270 6296 tablet_replica.cc:330] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377: stopping tablet replica
I20260619 16:54:28.143282 6295 raft_consensus.cc:2229] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:28.143316 6296 raft_consensus.cc:2229] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:28.143322 6295 raft_consensus.cc:2258] T ba2d63183fde431daa603909d61350b8 P d8e6aba81fb64f3cb8e52d70d122074e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:28.143455 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 5685
I20260619 16:54:28.143993 6296 raft_consensus.cc:2258] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:28.144313 6296 ts_tablet_manager.cc:1905] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:28.145262 6297 tablet_replica.cc:330] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42: stopping tablet replica
I20260619 16:54:28.145301 6297 raft_consensus.cc:2229] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:28.145334 6297 raft_consensus.cc:2258] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:28.145555 6296 ts_tablet_manager.cc:1918] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:28.145604 6296 log.cc:1196] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/ba2d63183fde431daa603909d61350b8
I20260619 16:54:28.145637 6297 ts_tablet_manager.cc:1905] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:28.145831 6296 ts_tablet_manager.cc:1939] T ba2d63183fde431daa603909d61350b8 P 9f3a049c850b48bd8bf962e2a9108377: Deleting consensus metadata
I20260619 16:54:28.146102 5708 catalog_manager.cc:4841] TS 9f3a049c850b48bd8bf962e2a9108377 (127.30.148.131:37105): tablet ba2d63183fde431daa603909d61350b8 (table test-table [id=469f910c92f94639aa906b5896230470]) successfully deleted
I20260619 16:54:28.147394 6297 ts_tablet_manager.cc:1918] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:28.147455 6297 log.cc:1196] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/ba2d63183fde431daa603909d61350b8
I20260619 16:54:28.147727 6297 ts_tablet_manager.cc:1939] T ba2d63183fde431daa603909d61350b8 P fb4bedcc98da432f8244bac04ed47c42: Deleting consensus metadata
I20260619 16:54:28.147998 5708 catalog_manager.cc:4841] TS fb4bedcc98da432f8244bac04ed47c42 (127.30.148.130:38313): tablet ba2d63183fde431daa603909d61350b8 (table test-table [id=469f910c92f94639aa906b5896230470]) successfully deleted
W20260619 16:54:28.149679 6257 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:28.149566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:44817 (local address 127.0.0.1:48528)
0619 16:54:28.149613 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:28.149637 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:44817: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 16:54:28.149835 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 5911
W20260619 16:54:28.149824 5710 catalog_manager.cc:4584] TS d8e6aba81fb64f3cb8e52d70d122074e (127.30.148.129:44817): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba2d63183fde431daa603909d61350b8: Network error: Client connection negotiation failed: client connection to 127.30.148.129:44817: connect: Connection reset by peer (error 104)
I20260619 16:54:28.155328 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 6047
I20260619 16:54:28.160068 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 5404
I20260619 16:54:28.164852 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 5592
2026-06-19T16:54:28Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6196 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:28 dist-test-slave-xb9n krb5kdc[6300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:28 dist-test-slave-xb9n krb5kdc[6300](info): set up 2 sockets
Jun 19 16:54:28 dist-test-slave-xb9n krb5kdc[6300](info): commencing operation
krb5kdc: starting...
W20260619 16:54:29.201773 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:29Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.190.keytab.
Entry for principal oryx/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.190.keytab.
I20260619 16:54:29.309906 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:43241
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.190.keytab
--principal=oryx/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:43241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:29.388518 6313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:29.388702 6313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:29.388731 6313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:29.389918 6313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:29.389958 6313 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 16:54:29.389972 6313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:29.390017 6313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:29.390034 6313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:29.390058 6313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:29.391542 6313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:43241
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.190.keytab
--principal=oryx/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:43241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:29.391788 6313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:29.392002 6313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:54:29.396620 6313 server_base.cc:1034] running on GCE node
W20260619 16:54:29.396596 6319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:29.396580 6318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:29.396591 6321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:29.397019 6313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:29.397295 6313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:29.398437 6313 hybrid_clock.cc:648] HybridClock initialized: now 1781888069398425 us; error 29 us; skew 500 ppm
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888069, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:29.400844 6313 init.cc:377] Logged in from keytab as oryx/127.30.148.190@KRBTEST.COM (short username oryx)
I20260619 16:54:29.401198 6313 webserver.cc:460] Webserver started at http://127.30.148.190:35883/ using document root <none> and password file <none>
I20260619 16:54:29.401440 6313 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:29.401485 6313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:29.401569 6313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:29.402493 6313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "982afb07b9144b649097ef27ce719666"
format_stamp: "Formatted at 2026-06-19 16:54:29 on dist-test-slave-xb9n"
server_key: "3618755e972773f47343a7c729af7279"
server_key_iv: "55eb51f2f58fe8b051ae451d1c190afd"
server_key_version: "encryptionkey@0"
I20260619 16:54:29.402787 6313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "982afb07b9144b649097ef27ce719666"
format_stamp: "Formatted at 2026-06-19 16:54:29 on dist-test-slave-xb9n"
server_key: "3618755e972773f47343a7c729af7279"
server_key_iv: "55eb51f2f58fe8b051ae451d1c190afd"
server_key_version: "encryptionkey@0"
I20260619 16:54:29.404045 6313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:54:29.404698 6328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:29.404874 6313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:29.404937 6313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "982afb07b9144b649097ef27ce719666"
format_stamp: "Formatted at 2026-06-19 16:54:29 on dist-test-slave-xb9n"
server_key: "3618755e972773f47343a7c729af7279"
server_key_iv: "55eb51f2f58fe8b051ae451d1c190afd"
server_key_version: "encryptionkey@0"
I20260619 16:54:29.405026 6313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:29.422080 6313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:29.424279 6313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:29.424443 6313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:29.428966 6313 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:43241
I20260619 16:54:29.429013 6390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:43241 every 8 connection(s)
I20260619 16:54:29.429311 6313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:29.429940 6391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:29.432088 6391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666: Bootstrap starting.
I20260619 16:54:29.432667 6391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:29.432973 6391 log.cc:824] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:29.433686 6391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666: No bootstrap required, opened a new log
I20260619 16:54:29.433851 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 6313
I20260619 16:54:29.433967 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:29.434147 31314 external_mini_cluster.cc:1428] Setting key 1c325f74bd0d59de59698ded03855853
I20260619 16:54:29.436403 6391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982afb07b9144b649097ef27ce719666" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43241 } }
I20260619 16:54:29.436573 6391 raft_consensus.cc:374] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:29.436612 6391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982afb07b9144b649097ef27ce719666, State: Initialized, Role: FOLLOWER
I20260619 16:54:29.436805 6391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982afb07b9144b649097ef27ce719666" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43241 } }
I20260619 16:54:29.436895 6391 raft_consensus.cc:388] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:29.436939 6391 raft_consensus.cc:482] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:29.436991 6391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:29.437638 6391 raft_consensus.cc:504] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982afb07b9144b649097ef27ce719666" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43241 } }
I20260619 16:54:29.437754 6391 leader_election.cc:302] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 982afb07b9144b649097ef27ce719666; no voters:
I20260619 16:54:29.437904 6391 leader_election.cc:288] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:29.437963 6396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:29.438133 6391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:29.438144 6396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [term 1 LEADER]: Becoming Leader. State: Replica: 982afb07b9144b649097ef27ce719666, State: Running, Role: LEADER
I20260619 16:54:29.438269 6396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982afb07b9144b649097ef27ce719666" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43241 } }
I20260619 16:54:29.438567 6396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 982afb07b9144b649097ef27ce719666. Latest consensus state: current_term: 1 leader_uuid: "982afb07b9144b649097ef27ce719666" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982afb07b9144b649097ef27ce719666" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43241 } } }
I20260619 16:54:29.438664 6396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:29.438586 6397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "982afb07b9144b649097ef27ce719666" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982afb07b9144b649097ef27ce719666" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43241 } } }
I20260619 16:54:29.438747 6397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:29.439172 6405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:29.439673 6405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:29.441336 6405 catalog_manager.cc:1347] Generated new cluster ID: 509ac03ea6ff414e80d647fb14e59d8b
I20260619 16:54:29.441411 6405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
I20260619 16:54:29.446308 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.434671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.30.148.190:43241)
0619 16:54:29.434860 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:29.434870 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:29.434904 (+ 34us) server_negotiation.cc:407] Connection header received
0619 16:54:29.436367 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:29.436376 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:29.436437 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:29.436586 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:29.437342 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.439065 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.440167 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.440751 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.440870 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.443155 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:29.443158 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:29.443159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:29.443170 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:29.444803 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.445180 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.445183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.445184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.445292 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.445522 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.445523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.445523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.445744 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:29.445917 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:29.445992 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 16:54:29.446127 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260619 16:54:29.446780 6405 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:29.447350 6405 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:29.451473 6405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666: Generated new TSK 0
I20260619 16:54:29.451637 6405 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab.
Entry for principal oryx/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab.
I20260619 16:54:29.479579 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:43241
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab
--principal=oryx/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:29.573571 6417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:29.573810 6417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:29.573839 6417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:29.574976 6417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:29.575017 6417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:29.575037 6417 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:54:29.575054 6417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:29.576504 6417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab
--principal=oryx/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.148.190:43241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:29.576721 6417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:29.576957 6417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:29.582823 6423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:29.582823 6425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:29.583026 6417 server_base.cc:1034] running on GCE node
W20260619 16:54:29.583103 6422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:29.583307 6417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:29.583552 6417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:29.584720 6417 hybrid_clock.cc:648] HybridClock initialized: now 1781888069584707 us; error 31 us; skew 500 ppm
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888069, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:29.587230 6417 init.cc:377] Logged in from keytab as oryx/127.30.148.129@KRBTEST.COM (short username oryx)
I20260619 16:54:29.587736 6417 webserver.cc:460] Webserver started at http://127.30.148.129:36407/ using document root <none> and password file <none>
I20260619 16:54:29.587975 6417 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:29.588039 6417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:29.588171 6417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:29.589149 6417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "a0ed48a38ec84ae0ad749395b8e0d741"
format_stamp: "Formatted at 2026-06-19 16:54:29 on dist-test-slave-xb9n"
server_key: "283192085adb939e0a3cbbd50e6eed55"
server_key_iv: "917459858e4a04e7ff9f487ebfa719ee"
server_key_version: "encryptionkey@0"
I20260619 16:54:29.589536 6417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "a0ed48a38ec84ae0ad749395b8e0d741"
format_stamp: "Formatted at 2026-06-19 16:54:29 on dist-test-slave-xb9n"
server_key: "283192085adb939e0a3cbbd50e6eed55"
server_key_iv: "917459858e4a04e7ff9f487ebfa719ee"
server_key_version: "encryptionkey@0"
I20260619 16:54:29.590853 6417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:29.591579 6432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:29.591756 6417 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:29.591845 6417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "a0ed48a38ec84ae0ad749395b8e0d741"
format_stamp: "Formatted at 2026-06-19 16:54:29 on dist-test-slave-xb9n"
server_key: "283192085adb939e0a3cbbd50e6eed55"
server_key_iv: "917459858e4a04e7ff9f487ebfa719ee"
server_key_version: "encryptionkey@0"
I20260619 16:54:29.591922 6417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:29.600639 6417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:29.602924 6417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:29.603077 6417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:29.603534 6417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:29.603595 6417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:29.603637 6417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:29.603669 6417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:29.610479 6417 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:38981
I20260619 16:54:29.610553 6547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:38981 every 8 connection(s)
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888069, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
I20260619 16:54:29.610872 6417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:29.613643 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.604047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:47375 (local address 127.30.148.190:43241)
0619 16:54:29.604257 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:29.604260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:29.604333 (+ 73us) server_negotiation.cc:407] Connection header received
0619 16:54:29.604892 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:29.604893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:29.604907 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:29.604986 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:29.605792 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.606625 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.608249 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.608658 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.608729 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.611257 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:29.611258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:29.611259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:29.611268 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:29.612503 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.612863 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.612863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.612864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.612910 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.613112 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.613112 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.613113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.613205 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:29.613254 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:29.613519 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 16:54:29.613582 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188}
I20260619 16:54:29.613866 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.604026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:43241 (local address 127.30.148.129:47375)
0619 16:54:29.604189 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:29.604252 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:29.604787 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:29.605034 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:29.605037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:29.605408 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:29.605688 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.605694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.606737 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.606738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.608148 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.608151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.608758 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.608758 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.608857 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.609348 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:29.609376 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:29.611128 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:29.612572 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:29.612578 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:29.612579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:29.612795 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:29.612969 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:29.612969 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:29.612970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:29.613059 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:29.613260 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:29.613264 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:29.613406 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 16:54:29.613505 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 16:54:29.613616 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260619 16:54:29.614338 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 6417
I20260619 16:54:29.614420 6548 heartbeater.cc:347] Connected to a master server at 127.30.148.190:43241
I20260619 16:54:29.614457 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:29.614486 6548 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:29.614619 31314 external_mini_cluster.cc:1428] Setting key 021bb82270f1b9b4201691ff2444c77f
I20260619 16:54:29.614656 6548 heartbeater.cc:511] Master 127.30.148.190:43241 requested a full tablet report, sending...
I20260619 16:54:29.615142 6345 ts_manager.cc:194] Registered new tserver with Master: a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129:38981)
I20260619 16:54:29.615978 6345 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:47375
WARNING: no policy specified for oryx/127.30.148.130@KRBTEST.COM; defaulting to no policy
I20260619 16:54:29.622617 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.616166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:48911 (local address 127.30.148.190:43241)
0619 16:54:29.616205 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:29.616207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:29.616504 (+ 297us) server_negotiation.cc:407] Connection header received
0619 16:54:29.616604 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:29.616605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:29.616615 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:29.616642 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:29.617071 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.617837 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.618916 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.619089 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.619160 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.620060 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:29.620061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:29.620061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:29.620068 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:29.621513 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.621868 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.621870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.621870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.621922 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.622127 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.622128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.622128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.622236 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:29.622312 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:29.622461 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 16:54:29.622535 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:29.622627 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.615423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:43241 (local address 127.30.148.129:48911)
0619 16:54:29.616445 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:29.616456 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:29.616527 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:29.616699 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:29.616700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:29.616919 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:29.616989 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.616990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.617936 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.617937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.618831 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.618834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.619174 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.619175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.619252 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.619676 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:29.619681 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:29.619953 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:29.621610 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:29.621611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:29.621611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:29.621782 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:29.621996 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:29.621997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:29.621997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:29.622054 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:29.622305 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:29.622306 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:29.622349 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 16:54:29.622437 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 16:54:29.622535 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"thread_start_us":61,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab.
Entry for principal oryx/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab.
I20260619 16:54:29.637246 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:43241
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab
--principal=oryx/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:29.716943 6557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:29.717115 6557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:29.717137 6557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:29.718303 6557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:29.718343 6557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:29.718362 6557 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:54:29.718407 6557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:29.719852 6557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab
--principal=oryx/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.148.190:43241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:29.720069 6557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:29.720274 6557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:29.724973 6563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:29.725009 6565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:29.724978 6562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:29.725217 6557 server_base.cc:1034] running on GCE node
I20260619 16:54:29.725440 6557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:29.725642 6557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:29.726771 6557 hybrid_clock.cc:648] HybridClock initialized: now 1781888069726761 us; error 27 us; skew 500 ppm
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888069, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:29.729060 6557 init.cc:377] Logged in from keytab as oryx/127.30.148.130@KRBTEST.COM (short username oryx)
I20260619 16:54:29.729489 6557 webserver.cc:460] Webserver started at http://127.30.148.130:33105/ using document root <none> and password file <none>
I20260619 16:54:29.729667 6557 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:29.729707 6557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:29.729789 6557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:29.730703 6557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "1ee47a7f03a747a1abe4df419b0f6d79"
format_stamp: "Formatted at 2026-06-19 16:54:29 on dist-test-slave-xb9n"
server_key: "0ea68978303e7bc25c4babc054656468"
server_key_iv: "8e11a58879dc4672134ce42e665847d5"
server_key_version: "encryptionkey@0"
I20260619 16:54:29.730984 6557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "1ee47a7f03a747a1abe4df419b0f6d79"
format_stamp: "Formatted at 2026-06-19 16:54:29 on dist-test-slave-xb9n"
server_key: "0ea68978303e7bc25c4babc054656468"
server_key_iv: "8e11a58879dc4672134ce42e665847d5"
server_key_version: "encryptionkey@0"
I20260619 16:54:29.732167 6557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:29.732789 6572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:29.732966 6557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:29.733031 6557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "1ee47a7f03a747a1abe4df419b0f6d79"
format_stamp: "Formatted at 2026-06-19 16:54:29 on dist-test-slave-xb9n"
server_key: "0ea68978303e7bc25c4babc054656468"
server_key_iv: "8e11a58879dc4672134ce42e665847d5"
server_key_version: "encryptionkey@0"
I20260619 16:54:29.733119 6557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:29.770870 6557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:29.772688 6557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:29.772845 6557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:29.773386 6557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:29.773419 6557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:29.773471 6557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:29.773501 6557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:29.780694 6557 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:35635
I20260619 16:54:29.780784 6687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:35635 every 8 connection(s)
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888069, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
I20260619 16:54:29.781111 6557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:54:29.782428 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 6557
I20260619 16:54:29.782560 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:29.782788 31314 external_mini_cluster.cc:1428] Setting key 248ca3521a1451e8766181ea7e4f4e42
I20260619 16:54:29.784224 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.773880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:34781 (local address 127.30.148.190:43241)
0619 16:54:29.773958 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:29.773960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:29.774123 (+ 163us) server_negotiation.cc:407] Connection header received
0619 16:54:29.774586 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:29.774587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:29.774599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:29.774679 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:29.775469 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.776316 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.777748 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.778136 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.778213 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.781565 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:29.781567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:29.781567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:29.781575 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:29.782844 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.783269 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.783271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.783271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.783328 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.783615 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.783616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.783616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.783728 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:29.783802 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:29.784058 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 16:54:29.784140 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 16:54:29.784394 6581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.773834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:43241 (local address 127.30.148.130:34781)
0619 16:54:29.773996 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:29.774033 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:29.774487 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:29.774743 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:29.774747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:29.775069 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:29.775374 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.775380 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.776415 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.776416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.777642 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.777645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.778285 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.778286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.778469 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.779599 (+ 1130us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:29.779608 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:29.781477 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:29.782943 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:29.782950 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:29.782952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:29.783198 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:29.783414 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:29.783415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:29.783415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:29.783529 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:29.783795 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:29.783800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:29.783931 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 16:54:29.784036 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 16:54:29.784179 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260619 16:54:29.784837 6688 heartbeater.cc:347] Connected to a master server at 127.30.148.190:43241
I20260619 16:54:29.784942 6688 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:29.785077 6688 heartbeater.cc:511] Master 127.30.148.190:43241 requested a full tablet report, sending...
I20260619 16:54:29.785424 6344 ts_manager.cc:194] Registered new tserver with Master: 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130:35635)
I20260619 16:54:29.785820 6344 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:34781
WARNING: no policy specified for oryx/127.30.148.131@KRBTEST.COM; defaulting to no policy
I20260619 16:54:29.791929 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.785959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:43241 (local address 127.30.148.130:50005)
0619 16:54:29.786118 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:29.786128 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:29.786230 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:29.786380 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:29.786381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:29.786589 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:29.786651 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.786652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.787229 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.787230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.788203 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.788214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.788540 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.788541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.788617 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.788997 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:29.789002 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:29.789419 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:29.790907 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:29.790908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:29.790908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:29.791113 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:29.791298 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:29.791299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:29.791299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:29.791370 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:29.791675 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:29.791676 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:29.791729 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 16:54:29.791787 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 16:54:29.791869 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":61,"threads_started":1}
I20260619 16:54:29.791935 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.785982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:50005 (local address 127.30.148.190:43241)
0619 16:54:29.786016 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:29.786018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:29.786174 (+ 156us) server_negotiation.cc:407] Connection header received
0619 16:54:29.786310 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:29.786311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:29.786321 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:29.786378 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:29.786732 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.787147 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.788291 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.788462 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.788515 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.789494 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:29.789496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:29.789496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:29.789501 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:29.790811 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.791187 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.791187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.791188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.791231 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.791472 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.791472 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.791472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.791601 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:29.791655 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:29.791819 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 16:54:29.791883 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab.
Entry for principal oryx/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab.
I20260619 16:54:29.805469 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:43241
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab
--principal=oryx/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:29.883070 6697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:29.883257 6697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:29.883277 6697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:29.884438 6697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:29.884476 6697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:29.884497 6697 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:54:29.884514 6697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:29.886020 6697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab
--principal=oryx/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.148.190:43241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:29.886198 6697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:29.886353 6697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:29.891100 6705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:29.891116 6702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:29.891111 6703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:29.891309 6697 server_base.cc:1034] running on GCE node
I20260619 16:54:29.891497 6697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:29.891672 6697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:29.892812 6697 hybrid_clock.cc:648] HybridClock initialized: now 1781888069892794 us; error 34 us; skew 500 ppm
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888069, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:29.895236 6697 init.cc:377] Logged in from keytab as oryx/127.30.148.131@KRBTEST.COM (short username oryx)
I20260619 16:54:29.895579 6697 webserver.cc:460] Webserver started at http://127.30.148.131:36581/ using document root <none> and password file <none>
I20260619 16:54:29.895758 6697 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:29.895812 6697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:29.895889 6697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:29.896793 6697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "9c9c07b39c7642d5bf0282c519f9e49b"
format_stamp: "Formatted at 2026-06-19 16:54:29 on dist-test-slave-xb9n"
server_key: "4406ae867267b2cab321006746c457a3"
server_key_iv: "c0e7c4ef97f55b4970d9741cfa43e1b6"
server_key_version: "encryptionkey@0"
I20260619 16:54:29.897069 6697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "9c9c07b39c7642d5bf0282c519f9e49b"
format_stamp: "Formatted at 2026-06-19 16:54:29 on dist-test-slave-xb9n"
server_key: "4406ae867267b2cab321006746c457a3"
server_key_iv: "c0e7c4ef97f55b4970d9741cfa43e1b6"
server_key_version: "encryptionkey@0"
I20260619 16:54:29.898384 6697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:29.899103 6712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:29.899281 6697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:29.899374 6697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "9c9c07b39c7642d5bf0282c519f9e49b"
format_stamp: "Formatted at 2026-06-19 16:54:29 on dist-test-slave-xb9n"
server_key: "4406ae867267b2cab321006746c457a3"
server_key_iv: "c0e7c4ef97f55b4970d9741cfa43e1b6"
server_key_version: "encryptionkey@0"
I20260619 16:54:29.899453 6697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:29.909263 6697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:29.910938 6697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:29.911088 6697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:29.911520 6697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:29.911579 6697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:29.911621 6697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:29.911648 6697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888069, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
I20260619 16:54:29.918860 6697 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:38917
I20260619 16:54:29.918942 6827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:38917 every 8 connection(s)
I20260619 16:54:29.919247 6697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:54:29.919414 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 6697
I20260619 16:54:29.919530 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:29.919735 31314 external_mini_cluster.cc:1428] Setting key 6e2c84ac584d98e0990b2a4d6cee7d89
I20260619 16:54:29.921484 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.911998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:49831 (local address 127.30.148.190:43241)
0619 16:54:29.912070 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:29.912073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:29.912307 (+ 234us) server_negotiation.cc:407] Connection header received
0619 16:54:29.912993 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:29.912994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:29.913003 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:29.913057 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:29.913986 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.914509 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.915614 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.915819 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.915904 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.918418 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:29.918419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:29.918420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:29.918427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:29.920044 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.920532 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.920533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.920534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.920593 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.920842 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.920843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.920843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.920956 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:29.921028 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:29.921295 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 16:54:29.921396 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 16:54:29.921636 6721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.911976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:43241 (local address 127.30.148.131:49831)
0619 16:54:29.912170 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:29.912216 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:29.912855 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:29.913131 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:29.913137 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:29.913558 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:29.913871 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.913877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.914617 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.914618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.915503 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.915506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.915913 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.915914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.916037 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.916530 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:29.916545 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:29.918321 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:29.920136 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:29.920143 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:29.920145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:29.920434 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:29.920660 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:29.920661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:29.920661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:29.920768 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:29.921040 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:29.921046 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:29.921168 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 16:54:29.921272 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 16:54:29.921432 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260619 16:54:29.922079 6828 heartbeater.cc:347] Connected to a master server at 127.30.148.190:43241
I20260619 16:54:29.922173 6828 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:29.922341 6828 heartbeater.cc:511] Master 127.30.148.190:43241 requested a full tablet report, sending...
I20260619 16:54:29.922725 6345 ts_manager.cc:194] Registered new tserver with Master: 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917)
I20260619 16:54:29.923120 6345 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:49831
I20260619 16:54:29.928314 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.923402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:57769 (local address 127.30.148.190:43241)
0619 16:54:29.923438 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:29.923440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:29.923526 (+ 86us) server_negotiation.cc:407] Connection header received
0619 16:54:29.923614 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:29.923615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:29.923622 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:29.923668 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:29.923986 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.924428 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.925061 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.925250 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.925325 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.926113 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:29.926114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:29.926114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:29.926120 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:29.927168 (+ 1048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.927564 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.927565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.927565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.927617 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.927826 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.927827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.927827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.927929 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:29.927998 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:29.928156 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 16:54:29.928231 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:29.928316 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.923356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:43241 (local address 127.30.148.131:57769)
0619 16:54:29.923472 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:29.923481 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:29.923543 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:29.923674 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:29.923675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:29.923857 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:29.923912 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.923913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.924507 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.924508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.924993 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.924994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.925327 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.925328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.925429 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.925762 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:29.925766 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:29.926009 (+ 243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:29.927296 (+ 1287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:29.927296 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:29.927297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:29.927462 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:29.927689 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:29.927690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:29.927691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:29.927755 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:29.928002 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:29.928003 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:29.928045 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 16:54:29.928132 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 16:54:29.928228 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260619 16:54:29.931248 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888069, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 16:54:29.951216 6395 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:29.946352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56718 (local address 127.30.148.190:43241)
0619 16:54:29.946389 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:29.946393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:29.946467 (+ 74us) server_negotiation.cc:407] Connection header received
0619 16:54:29.946543 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:29.946544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:29.946552 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:29.946599 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:29.947075 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.947560 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.948214 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.948379 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.948427 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.951126 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56718: BlockingRecv error: recv got EOF from 127.0.0.1:56718 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Jun 19 16:54:29 dist-test-slave-xb9n krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888069, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
I20260619 16:54:29.958091 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.952096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56728 (local address 127.30.148.190:43241)
0619 16:54:29.952148 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:29.952150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:29.952221 (+ 71us) server_negotiation.cc:407] Connection header received
0619 16:54:29.952277 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:29.952278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:29.952285 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:29.952333 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:29.952764 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.953186 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.953871 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.954036 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.954087 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.956099 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:29.956100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:29.956100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:29.956106 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:29.957221 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.957560 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.957561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.957561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.957608 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:29.957763 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:29.957763 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:29.957764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:29.957843 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:29.957909 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:29.957991 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 16:54:29.958040 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 16:54:29.959748 6345 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:29.961259 6345 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:29.967948 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.963638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:38917 (local address 127.0.0.1:60382)
0619 16:54:29.964031 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:29.964041 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:29.964095 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:29.964296 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:29.964297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:29.964299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:29.964347 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.964349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.965548 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.965549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.966674 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.966677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.967618 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.967619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.967701 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.967724 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 16:54:29.967798 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 16:54:29.967847 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260619 16:54:29.967957 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.963832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60382 (local address 127.30.148.131:38917)
0619 16:54:29.963998 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:29.964004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:29.964109 (+ 105us) server_negotiation.cc:407] Connection header received
0619 16:54:29.964152 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:29.964154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:29.964170 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:29.964255 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:29.964407 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.965480 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.966765 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.967491 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.967641 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.967691 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:29.967822 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 16:54:29.967861 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":80,"threads_started":1}
I20260619 16:54:29.968130 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.963458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:38981 (local address 127.0.0.1:49246)
0619 16:54:29.963849 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:29.963859 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:29.963903 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:29.964448 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:29.964449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:29.964451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:29.964482 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.964483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.965317 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.965318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.966676 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.966678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.967690 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.967691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.967761 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.967764 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:29.967834 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 16:54:29.967875 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":40,"thread_start_us":80,"threads_started":1}
I20260619 16:54:29.968178 6854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.963658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49246 (local address 127.30.148.129:38981)
0619 16:54:29.964237 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:29.964242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:29.964252 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:29.964290 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:29.964292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:29.964306 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:29.964384 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:29.964555 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.965225 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.966768 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.967492 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.967593 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.967655 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:29.967994 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 16:54:29.968076 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":72,"threads_started":1}
I20260619 16:54:29.968696 6850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.963458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:35635 (local address 127.0.0.1:45162)
0619 16:54:29.963606 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:29.963642 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:29.963700 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:29.963963 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:29.963966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:29.963968 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:29.964041 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.964043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.966775 (+ 2732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.966776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.967425 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:29.967427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.968390 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.968391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:29.968520 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.968524 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:29.968582 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 16:54:29.968629 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":51,"threads_started":1}
I20260619 16:54:29.968750 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:29.963562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.30.148.130:35635)
0619 16:54:29.963777 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:29.963782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:29.963793 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:54:29.963829 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:29.963831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:29.963852 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:29.963944 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:29.964128 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.965464 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.967548 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:29.968299 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:29.968393 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:29.968444 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:29.968607 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 16:54:29.968648 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":87,"threads_started":1}
I20260619 16:54:29.968899 6482 tablet_service.cc:1467] Processing CreateTablet for tablet 21e6c5719fe84b0c9ded6e1eda6b2c32 (DEFAULT_TABLE table=test-table [id=4c926b293fa5432db6d2a3fe64500ba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:29.969271 6482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21e6c5719fe84b0c9ded6e1eda6b2c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:29.969313 6622 tablet_service.cc:1467] Processing CreateTablet for tablet 21e6c5719fe84b0c9ded6e1eda6b2c32 (DEFAULT_TABLE table=test-table [id=4c926b293fa5432db6d2a3fe64500ba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:29.969476 6762 tablet_service.cc:1467] Processing CreateTablet for tablet 21e6c5719fe84b0c9ded6e1eda6b2c32 (DEFAULT_TABLE table=test-table [id=4c926b293fa5432db6d2a3fe64500ba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:29.969671 6622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21e6c5719fe84b0c9ded6e1eda6b2c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:29.969851 6762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21e6c5719fe84b0c9ded6e1eda6b2c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:29.972450 6856 tablet_bootstrap.cc:492] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b: Bootstrap starting.
I20260619 16:54:29.973095 6856 tablet_bootstrap.cc:654] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:29.973487 6856 log.cc:824] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:29.973482 6857 tablet_bootstrap.cc:492] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741: Bootstrap starting.
I20260619 16:54:29.973472 6858 tablet_bootstrap.cc:492] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79: Bootstrap starting.
I20260619 16:54:29.974090 6857 tablet_bootstrap.cc:654] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:29.974090 6858 tablet_bootstrap.cc:654] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:29.974192 6856 tablet_bootstrap.cc:492] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b: No bootstrap required, opened a new log
I20260619 16:54:29.974259 6856 ts_tablet_manager.cc:1397] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:29.974421 6857 log.cc:824] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:29.974421 6858 log.cc:824] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:29.975119 6858 tablet_bootstrap.cc:492] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79: No bootstrap required, opened a new log
I20260619 16:54:29.975211 6858 ts_tablet_manager.cc:1397] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:29.975355 6857 tablet_bootstrap.cc:492] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741: No bootstrap required, opened a new log
I20260619 16:54:29.975415 6857 ts_tablet_manager.cc:1397] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:29.976112 6856 raft_consensus.cc:348] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } }
I20260619 16:54:29.976286 6856 raft_consensus.cc:374] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:29.976331 6856 raft_consensus.cc:729] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c9c07b39c7642d5bf0282c519f9e49b, State: Initialized, Role: FOLLOWER
I20260619 16:54:29.976444 6856 consensus_queue.cc:260] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } }
I20260619 16:54:29.976720 6856 ts_tablet_manager.cc:1428] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 16:54:29.976729 6828 heartbeater.cc:503] Master 127.30.148.190:43241 was elected leader, sending a full tablet report...
I20260619 16:54:29.977159 6858 raft_consensus.cc:348] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } }
I20260619 16:54:29.977326 6858 raft_consensus.cc:374] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:29.977396 6858 raft_consensus.cc:729] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee47a7f03a747a1abe4df419b0f6d79, State: Initialized, Role: FOLLOWER
I20260619 16:54:29.977311 6857 raft_consensus.cc:348] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } }
I20260619 16:54:29.977481 6857 raft_consensus.cc:374] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:29.977523 6857 raft_consensus.cc:729] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ed48a38ec84ae0ad749395b8e0d741, State: Initialized, Role: FOLLOWER
I20260619 16:54:29.977526 6858 consensus_queue.cc:260] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } }
I20260619 16:54:29.977634 6857 consensus_queue.cc:260] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } }
I20260619 16:54:29.977766 6858 ts_tablet_manager.cc:1428] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:54:29.977880 6857 ts_tablet_manager.cc:1428] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 16:54:29.977893 6688 heartbeater.cc:503] Master 127.30.148.190:43241 was elected leader, sending a full tablet report...
I20260619 16:54:29.978005 6548 heartbeater.cc:503] Master 127.30.148.190:43241 was elected leader, sending a full tablet report...
W20260619 16:54:30.031800 6689 tablet.cc:2369] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:54:30.084427 6862 raft_consensus.cc:482] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:30.084550 6862 raft_consensus.cc:504] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } }
I20260619 16:54:30.084928 6862 leader_election.cc:288] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130:35635), a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129:38981)
I20260619 16:54:30.088857 6854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.085230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:49821 (local address 127.30.148.129:38981)
0619 16:54:30.085277 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.085279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.085430 (+ 151us) server_negotiation.cc:407] Connection header received
0619 16:54:30.085461 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.085462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.085471 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.085565 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:30.085689 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.086413 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.087912 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.088476 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.088560 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.088570 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.088748 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.088788 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:54:30.088857 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.085243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:35927 (local address 127.30.148.130:35635)
0619 16:54:30.085285 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.085287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.085295 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:30.085323 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.085325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.085335 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.085452 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:30.085641 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.086417 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.087912 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.088466 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.088560 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.088570 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.088748 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.088787 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:54:30.088883 6721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.085123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:35635 (local address 127.30.148.131:35927)
0619 16:54:30.085170 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:30.085183 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:30.085228 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:30.085459 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:30.085460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:30.085462 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:30.085560 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.085562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.086496 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.086496 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.087816 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.087820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.088571 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.088572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.088662 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.088664 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:54:30.088723 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 16:54:30.088766 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":43}
I20260619 16:54:30.088882 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.085123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:38981 (local address 127.30.148.131:49821)
0619 16:54:30.085340 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:30.085348 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:30.085415 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:30.085547 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:30.085547 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:30.085548 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:30.085619 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.085620 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.086496 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.086496 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.087816 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.087820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.088578 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.088579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.088659 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.088663 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:30.088723 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 16:54:30.088766 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":110,"thread_start_us":106,"threads_started":1}
I20260619 16:54:30.089301 6502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21e6c5719fe84b0c9ded6e1eda6b2c32" candidate_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" is_pre_election: true
I20260619 16:54:30.089300 6632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21e6c5719fe84b0c9ded6e1eda6b2c32" candidate_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" is_pre_election: true
I20260619 16:54:30.089486 6502 raft_consensus.cc:2454] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c9c07b39c7642d5bf0282c519f9e49b in term 0.
I20260619 16:54:30.089488 6632 raft_consensus.cc:2454] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c9c07b39c7642d5bf0282c519f9e49b in term 0.
I20260619 16:54:30.089753 6713 leader_election.cc:302] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c9c07b39c7642d5bf0282c519f9e49b, a0ed48a38ec84ae0ad749395b8e0d741; no voters:
I20260619 16:54:30.089926 6862 raft_consensus.cc:2781] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:30.089973 6862 raft_consensus.cc:482] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:30.089989 6862 raft_consensus.cc:3037] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.090956 6862 raft_consensus.cc:504] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } }
I20260619 16:54:30.091094 6862 leader_election.cc:288] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [CANDIDATE]: Term 1 election: Requested vote from peers 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130:35635), a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129:38981)
I20260619 16:54:30.091235 6502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21e6c5719fe84b0c9ded6e1eda6b2c32" candidate_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ed48a38ec84ae0ad749395b8e0d741"
I20260619 16:54:30.091262 6632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21e6c5719fe84b0c9ded6e1eda6b2c32" candidate_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ee47a7f03a747a1abe4df419b0f6d79"
I20260619 16:54:30.091311 6502 raft_consensus.cc:3037] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.091352 6632 raft_consensus.cc:3037] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.092022 6632 raft_consensus.cc:2454] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c9c07b39c7642d5bf0282c519f9e49b in term 1.
I20260619 16:54:30.092041 6502 raft_consensus.cc:2454] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c9c07b39c7642d5bf0282c519f9e49b in term 1.
I20260619 16:54:30.092192 6713 leader_election.cc:302] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c9c07b39c7642d5bf0282c519f9e49b, a0ed48a38ec84ae0ad749395b8e0d741; no voters:
I20260619 16:54:30.092289 6862 raft_consensus.cc:2781] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:30.092449 6862 raft_consensus.cc:686] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 LEADER]: Becoming Leader. State: Replica: 9c9c07b39c7642d5bf0282c519f9e49b, State: Running, Role: LEADER
I20260619 16:54:30.092558 6862 consensus_queue.cc:237] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } }
I20260619 16:54:30.093413 6344 catalog_manager.cc:5495] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131). New cstate: current_term: 1 leader_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } health_report { overall_health: UNKNOWN } } }
W20260619 16:54:30.111671 6549 tablet.cc:2369] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:54:30.113080 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.110727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60392 (local address 127.30.148.131:38917)
0619 16:54:30.110783 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.110785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.110793 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:30.110827 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.110828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.110850 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.110918 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:30.111044 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.111486 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.112314 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.112513 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.112605 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.112776 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:30.112871 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:30.112956 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.112984 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.113002 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 16:54:30.114681 6502 raft_consensus.cc:1261] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Refusing update from remote peer 9c9c07b39c7642d5bf0282c519f9e49b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:30.114681 6632 raft_consensus.cc:1261] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Refusing update from remote peer 9c9c07b39c7642d5bf0282c519f9e49b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:30.115044 6866 consensus_queue.cc:1035] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:30.115180 6862 consensus_queue.cc:1035] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:30.117431 6871 mvcc.cc:205] Tried to move back new op lower bound from 7298613535187365888 to 7298613535099731968. Current Snapshot: MvccSnapshot[applied={T|T < 7298613535187365888}]
I20260619 16:54:30.117772 6870 mvcc.cc:205] Tried to move back new op lower bound from 7298613535187365888 to 7298613535099731968. Current Snapshot: MvccSnapshot[applied={T|T < 7298613535187365888}]
I20260619 16:54:30.119824 6344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:30.119923 6344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:30.120673 6344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666: Sending DeleteTablet for 3 replicas of tablet 21e6c5719fe84b0c9ded6e1eda6b2c32
I20260619 16:54:30.121012 6482 tablet_service.cc:1514] Processing DeleteTablet for tablet 21e6c5719fe84b0c9ded6e1eda6b2c32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:30 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:49246
I20260619 16:54:30.121047 6762 tablet_service.cc:1514] Processing DeleteTablet for tablet 21e6c5719fe84b0c9ded6e1eda6b2c32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:30 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:60382
I20260619 16:54:30.121045 6622 tablet_service.cc:1514] Processing DeleteTablet for tablet 21e6c5719fe84b0c9ded6e1eda6b2c32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:30 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:45162
I20260619 16:54:30.121258 6879 tablet_replica.cc:330] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b: stopping tablet replica
I20260619 16:54:30.121264 6878 tablet_replica.cc:330] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741: stopping tablet replica
I20260619 16:54:30.121373 6879 raft_consensus.cc:2229] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:30.121373 6878 raft_consensus.cc:2229] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:30.121472 6878 raft_consensus.cc:2258] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:30.121490 6879 raft_consensus.cc:2258] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:30.121714 6344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 16:54:30.121819 6880 tablet_replica.cc:330] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79: stopping tablet replica
W20260619 16:54:30.121862 6344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:30.121901 6880 raft_consensus.cc:2229] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:30.121994 6880 raft_consensus.cc:2258] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:30.122320 6878 ts_tablet_manager.cc:1905] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:30.122465 6880 ts_tablet_manager.cc:1905] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:30.123343 6482 tablet_service.cc:1467] Processing CreateTablet for tablet d2b675655a5e418eba07cca0c5a462c7 (DEFAULT_TABLE table=test-table [id=78a715ddd4e048fb8706a5e2a05edc16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:30.123696 6622 tablet_service.cc:1467] Processing CreateTablet for tablet d2b675655a5e418eba07cca0c5a462c7 (DEFAULT_TABLE table=test-table [id=78a715ddd4e048fb8706a5e2a05edc16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:30.123837 6622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2b675655a5e418eba07cca0c5a462c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:30.124065 6880 ts_tablet_manager.cc:1918] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:30.124140 6880 log.cc:1196] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/21e6c5719fe84b0c9ded6e1eda6b2c32
I20260619 16:54:30.124392 6880 ts_tablet_manager.cc:1939] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 1ee47a7f03a747a1abe4df419b0f6d79: Deleting consensus metadata
I20260619 16:54:30.124820 6332 catalog_manager.cc:4841] TS 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130:35635): tablet 21e6c5719fe84b0c9ded6e1eda6b2c32 (table test-table [id=4c926b293fa5432db6d2a3fe64500ba3]) successfully deleted
I20260619 16:54:30.124965 6858 tablet_bootstrap.cc:492] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79: Bootstrap starting.
I20260619 16:54:30.125006 6762 tablet_service.cc:1467] Processing CreateTablet for tablet d2b675655a5e418eba07cca0c5a462c7 (DEFAULT_TABLE table=test-table [id=78a715ddd4e048fb8706a5e2a05edc16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:30.125123 6762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2b675655a5e418eba07cca0c5a462c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:30.125350 6858 tablet_bootstrap.cc:654] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:30.125983 6858 tablet_bootstrap.cc:492] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79: No bootstrap required, opened a new log
I20260619 16:54:30.126034 6858 ts_tablet_manager.cc:1397] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:30.126169 6858 raft_consensus.cc:348] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.126238 6858 raft_consensus.cc:374] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:30.126262 6858 raft_consensus.cc:729] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee47a7f03a747a1abe4df419b0f6d79, State: Initialized, Role: FOLLOWER
I20260619 16:54:30.126310 6858 consensus_queue.cc:260] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.126374 6856 tablet_bootstrap.cc:492] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b: Bootstrap starting.
I20260619 16:54:30.126396 6858 ts_tablet_manager.cc:1428] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:30.126835 6856 tablet_bootstrap.cc:654] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:30.127300 6482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2b675655a5e418eba07cca0c5a462c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:30.128237 6857 tablet_bootstrap.cc:492] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741: Bootstrap starting.
I20260619 16:54:30.128582 6857 tablet_bootstrap.cc:654] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:30.128911 6878 ts_tablet_manager.cc:1918] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:30.129004 6878 log.cc:1196] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/wal/wals/21e6c5719fe84b0c9ded6e1eda6b2c32
I20260619 16:54:30.129236 6857 tablet_bootstrap.cc:492] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741: No bootstrap required, opened a new log
I20260619 16:54:30.129287 6857 ts_tablet_manager.cc:1397] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:30.129479 6857 raft_consensus.cc:348] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.129552 6857 raft_consensus.cc:374] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:30.129578 6857 raft_consensus.cc:729] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ed48a38ec84ae0ad749395b8e0d741, State: Initialized, Role: FOLLOWER
I20260619 16:54:30.129621 6878 ts_tablet_manager.cc:1939] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P a0ed48a38ec84ae0ad749395b8e0d741: Deleting consensus metadata
I20260619 16:54:30.129637 6857 consensus_queue.cc:260] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.129732 6856 tablet_bootstrap.cc:492] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b: No bootstrap required, opened a new log
I20260619 16:54:30.129743 6857 ts_tablet_manager.cc:1428] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:30.129776 6856 ts_tablet_manager.cc:1397] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 16:54:30.129911 6856 raft_consensus.cc:348] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.129967 6856 raft_consensus.cc:374] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:30.129987 6856 raft_consensus.cc:729] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c9c07b39c7642d5bf0282c519f9e49b, State: Initialized, Role: FOLLOWER
I20260619 16:54:30.130029 6856 consensus_queue.cc:260] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.130024 6329 catalog_manager.cc:4841] TS a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129:38981): tablet 21e6c5719fe84b0c9ded6e1eda6b2c32 (table test-table [id=4c926b293fa5432db6d2a3fe64500ba3]) successfully deleted
I20260619 16:54:30.130100 6856 ts_tablet_manager.cc:1428] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:30.130366 6879 ts_tablet_manager.cc:1905] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:30.131734 6879 ts_tablet_manager.cc:1918] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:30.131803 6879 log.cc:1196] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/21e6c5719fe84b0c9ded6e1eda6b2c32
I20260619 16:54:30.132040 6879 ts_tablet_manager.cc:1939] T 21e6c5719fe84b0c9ded6e1eda6b2c32 P 9c9c07b39c7642d5bf0282c519f9e49b: Deleting consensus metadata
I20260619 16:54:30.132391 6332 catalog_manager.cc:4841] TS 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917): tablet 21e6c5719fe84b0c9ded6e1eda6b2c32 (table test-table [id=4c926b293fa5432db6d2a3fe64500ba3]) successfully deleted
W20260619 16:54:30.169759 6829 tablet.cc:2369] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:54:30.207247 6864 raft_consensus.cc:482] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:30.207345 6864 raft_consensus.cc:504] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.207679 6864 leader_election.cc:288] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130:35635), 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917)
I20260619 16:54:30.211352 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.208105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:37211 (local address 127.30.148.131:38917)
0619 16:54:30.208163 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.208165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.208266 (+ 101us) server_negotiation.cc:407] Connection header received
0619 16:54:30.208361 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.208363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.208372 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.208399 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:30.208610 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.209303 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.210584 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.210964 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.211053 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.211065 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.211251 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.211286 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:54:30.211365 6885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.208054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:38917 (local address 127.30.148.129:37211)
0619 16:54:30.208204 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:30.208214 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:30.208279 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:30.208486 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:30.208486 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:30.208488 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:30.208556 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.208558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.209419 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.209420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.210502 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.210504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.211069 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.211071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.211153 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.211160 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 16:54:30.211239 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 16:54:30.211284 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260619 16:54:30.211428 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.207953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:60753 (local address 127.30.148.130:35635)
0619 16:54:30.208003 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.208006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.208111 (+ 105us) server_negotiation.cc:407] Connection header received
0619 16:54:30.208243 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.208245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.208256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.208325 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:30.208483 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.209034 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.210458 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.211043 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.211120 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.211128 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.211322 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.211362 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 16:54:30.211637 6782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2b675655a5e418eba07cca0c5a462c7" candidate_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" is_pre_election: true
I20260619 16:54:30.211767 6782 raft_consensus.cc:2454] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ed48a38ec84ae0ad749395b8e0d741 in term 0.
I20260619 16:54:30.211840 6884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.207853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:35635 (local address 127.30.148.129:60753)
0619 16:54:30.208034 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:30.208046 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:30.208142 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:30.208330 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:30.208331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:30.208333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:30.208405 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.208407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.209118 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.209119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.210356 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.210358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.211163 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.211164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.211237 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.211241 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:30.211688 (+ 447us) client_negotiation.cc:240] Negotiation successful
0619 16:54:30.211737 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":95,"threads_started":1}
I20260619 16:54:30.211980 6436 leader_election.cc:302] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c9c07b39c7642d5bf0282c519f9e49b, a0ed48a38ec84ae0ad749395b8e0d741; no voters:
I20260619 16:54:30.212126 6864 raft_consensus.cc:2781] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:30.212177 6864 raft_consensus.cc:482] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:30.212198 6864 raft_consensus.cc:3037] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.212205 6632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2b675655a5e418eba07cca0c5a462c7" candidate_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" is_pre_election: true
I20260619 16:54:30.212284 6632 raft_consensus.cc:2454] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ed48a38ec84ae0ad749395b8e0d741 in term 0.
I20260619 16:54:30.213076 6864 raft_consensus.cc:504] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.213224 6864 leader_election.cc:288] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [CANDIDATE]: Term 1 election: Requested vote from peers 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130:35635), 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917)
I20260619 16:54:30.213404 6632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2b675655a5e418eba07cca0c5a462c7" candidate_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ee47a7f03a747a1abe4df419b0f6d79"
I20260619 16:54:30.213444 6782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2b675655a5e418eba07cca0c5a462c7" candidate_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c9c07b39c7642d5bf0282c519f9e49b"
I20260619 16:54:30.213478 6632 raft_consensus.cc:3037] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.213506 6782 raft_consensus.cc:3037] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.214064 6632 raft_consensus.cc:2454] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ed48a38ec84ae0ad749395b8e0d741 in term 1.
I20260619 16:54:30.214249 6436 leader_election.cc:302] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ee47a7f03a747a1abe4df419b0f6d79, a0ed48a38ec84ae0ad749395b8e0d741; no voters:
I20260619 16:54:30.214330 6782 raft_consensus.cc:2454] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ed48a38ec84ae0ad749395b8e0d741 in term 1.
I20260619 16:54:30.214350 6864 raft_consensus.cc:2781] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:30.214442 6864 raft_consensus.cc:686] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 LEADER]: Becoming Leader. State: Replica: a0ed48a38ec84ae0ad749395b8e0d741, State: Running, Role: LEADER
I20260619 16:54:30.214506 6864 consensus_queue.cc:237] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.215157 6344 catalog_manager.cc:5495] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129). New cstate: current_term: 1 leader_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } health_report { overall_health: UNKNOWN } } }
Jun 19 16:54:30 dist-test-slave-xb9n krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888069, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.190@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
I20260619 16:54:30.244397 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.237960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:43241 (local address 127.0.0.1:56736)
0619 16:54:30.238143 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:30.238153 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:30.238204 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:30.238326 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:30.238326 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:30.238658 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:30.238716 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.238750 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.239369 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.239370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.239933 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.239934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.240229 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.240230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.240267 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.240797 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:30.240801 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:30.242226 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:30.243560 (+ 1334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:30.243565 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:30.243566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:30.243741 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:30.243913 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:30.243913 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:30.243913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:30.243963 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:30.244154 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:30.244159 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:30.244196 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 16:54:30.244246 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 16:54:30.244324 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":88,"threads_started":1}
I20260619 16:54:30.244475 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.238030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.30.148.190:43241)
0619 16:54:30.238083 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.238084 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.238240 (+ 156us) server_negotiation.cc:407] Connection header received
0619 16:54:30.238259 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.238260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.238268 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.238308 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:30.238817 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.239309 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.239996 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.240169 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.240222 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.242308 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:30.242309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:30.242310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:30.242315 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:30.243486 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:30.243802 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:30.243803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:30.243803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:30.243858 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:30.244014 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:30.244015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:30.244015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:30.244103 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:30.244152 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.244260 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.244365 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":33}
I20260619 16:54:30.246219 6344 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:56736:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 16:54:30.246389 6344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:30.247673 6622 tablet_service.cc:1467] Processing CreateTablet for tablet a99753b44da646cf9d604d9670d2fa67 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0accc39c14a849419f9b69d8d91ba9e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:54:30.247707 6762 tablet_service.cc:1467] Processing CreateTablet for tablet a99753b44da646cf9d604d9670d2fa67 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0accc39c14a849419f9b69d8d91ba9e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:54:30.247712 6482 tablet_service.cc:1467] Processing CreateTablet for tablet a99753b44da646cf9d604d9670d2fa67 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0accc39c14a849419f9b69d8d91ba9e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:54:30.247824 6482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a99753b44da646cf9d604d9670d2fa67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:30.247830 6622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a99753b44da646cf9d604d9670d2fa67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:30.247829 6762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a99753b44da646cf9d604d9670d2fa67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:30.249205 6858 tablet_bootstrap.cc:492] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79: Bootstrap starting.
I20260619 16:54:30.249209 6856 tablet_bootstrap.cc:492] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b: Bootstrap starting.
I20260619 16:54:30.249219 6857 tablet_bootstrap.cc:492] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: Bootstrap starting.
I20260619 16:54:30.249534 6856 tablet_bootstrap.cc:654] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:30.249650 6858 tablet_bootstrap.cc:654] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:30.249650 6857 tablet_bootstrap.cc:654] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:30.250313 6858 tablet_bootstrap.cc:492] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79: No bootstrap required, opened a new log
I20260619 16:54:30.250367 6858 ts_tablet_manager.cc:1397] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:30.250509 6857 tablet_bootstrap.cc:492] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: No bootstrap required, opened a new log
I20260619 16:54:30.250559 6857 ts_tablet_manager.cc:1397] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:30.250540 6858 raft_consensus.cc:348] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.250631 6858 raft_consensus.cc:374] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:30.250669 6858 raft_consensus.cc:729] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee47a7f03a747a1abe4df419b0f6d79, State: Initialized, Role: FOLLOWER
I20260619 16:54:30.250697 6857 raft_consensus.cc:348] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.250727 6858 consensus_queue.cc:260] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.250771 6857 raft_consensus.cc:374] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:30.250803 6857 raft_consensus.cc:729] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ed48a38ec84ae0ad749395b8e0d741, State: Initialized, Role: FOLLOWER
I20260619 16:54:30.250820 6858 ts_tablet_manager.cc:1428] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:30.250844 6857 consensus_queue.cc:260] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.250952 6857 ts_tablet_manager.cc:1428] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:30.250974 6863 tablet_replica.cc:439] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } }
I20260619 16:54:30.251055 6863 tablet_replica.cc:442] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:30.251185 6864 tablet_replica.cc:439] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } }
I20260619 16:54:30.251298 6864 tablet_replica.cc:442] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:30.251698 6856 tablet_bootstrap.cc:492] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b: No bootstrap required, opened a new log
I20260619 16:54:30.251754 6856 ts_tablet_manager.cc:1397] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 16:54:30.251875 6856 raft_consensus.cc:348] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.251936 6856 raft_consensus.cc:374] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:30.251950 6856 raft_consensus.cc:729] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c9c07b39c7642d5bf0282c519f9e49b, State: Initialized, Role: FOLLOWER
I20260619 16:54:30.252004 6856 consensus_queue.cc:260] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.252122 6856 ts_tablet_manager.cc:1428] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:30.252189 6866 tablet_replica.cc:439] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } }
I20260619 16:54:30.252305 6866 tablet_replica.cc:442] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:30.364481 6864 raft_consensus.cc:482] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:30.364595 6864 raft_consensus.cc:504] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.364771 6864 leader_election.cc:288] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130:35635), 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917)
I20260619 16:54:30.365006 6632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a99753b44da646cf9d604d9670d2fa67" candidate_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" is_pre_election: true
I20260619 16:54:30.365106 6632 raft_consensus.cc:2454] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ed48a38ec84ae0ad749395b8e0d741 in term 0.
I20260619 16:54:30.365046 6782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a99753b44da646cf9d604d9670d2fa67" candidate_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" is_pre_election: true
I20260619 16:54:30.365162 6782 raft_consensus.cc:2454] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0ed48a38ec84ae0ad749395b8e0d741 in term 0.
I20260619 16:54:30.365260 6436 leader_election.cc:302] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ee47a7f03a747a1abe4df419b0f6d79, a0ed48a38ec84ae0ad749395b8e0d741; no voters:
I20260619 16:54:30.365415 6864 raft_consensus.cc:2781] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:30.365456 6864 raft_consensus.cc:482] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:30.365476 6864 raft_consensus.cc:3037] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.366123 6864 raft_consensus.cc:504] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.366241 6864 leader_election.cc:288] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [CANDIDATE]: Term 1 election: Requested vote from peers 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130:35635), 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917)
I20260619 16:54:30.366451 6782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a99753b44da646cf9d604d9670d2fa67" candidate_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c9c07b39c7642d5bf0282c519f9e49b"
I20260619 16:54:30.366451 6632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a99753b44da646cf9d604d9670d2fa67" candidate_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ee47a7f03a747a1abe4df419b0f6d79"
I20260619 16:54:30.366526 6632 raft_consensus.cc:3037] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.366526 6782 raft_consensus.cc:3037] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.367268 6632 raft_consensus.cc:2454] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ed48a38ec84ae0ad749395b8e0d741 in term 1.
I20260619 16:54:30.367303 6782 raft_consensus.cc:2454] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0ed48a38ec84ae0ad749395b8e0d741 in term 1.
I20260619 16:54:30.367435 6436 leader_election.cc:302] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ee47a7f03a747a1abe4df419b0f6d79, a0ed48a38ec84ae0ad749395b8e0d741; no voters:
I20260619 16:54:30.367560 6864 raft_consensus.cc:2781] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:30.367614 6864 raft_consensus.cc:686] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 LEADER]: Becoming Leader. State: Replica: a0ed48a38ec84ae0ad749395b8e0d741, State: Running, Role: LEADER
I20260619 16:54:30.367663 6864 consensus_queue.cc:237] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.368000 6886 tablet_replica.cc:439] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: TxnStatusTablet state changed. Reason: New leader a0ed48a38ec84ae0ad749395b8e0d741. Latest consensus state: current_term: 1 leader_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } }
I20260619 16:54:30.368083 6886 tablet_replica.cc:442] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:54:30.368265 6898 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 16:54:30.368369 6898 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 16:54:30.368295 6344 catalog_manager.cc:5495] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129). New cstate: current_term: 1 leader_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } health_report { overall_health: UNKNOWN } } }
I20260619 16:54:30.396538 6854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.393767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49258 (local address 127.30.148.129:38981)
0619 16:54:30.393834 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.393835 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.393844 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:54:30.393888 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.393889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.393905 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.393962 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:30.394133 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.394663 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.395780 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.395960 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.396018 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.396214 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:30.396308 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:30.396362 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.396433 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.396449 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 16:54:30.396561 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.393264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:38981 (local address 127.0.0.1:49258)
0619 16:54:30.393373 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:30.393385 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:30.393437 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:30.393996 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:30.393997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:30.393999 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:54:30.394058 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.394061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.394734 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.394735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.395702 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.395705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.396040 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.396041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.396109 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.396125 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:54:30.396369 (+ 244us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:54:30.396376 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 16:54:30.396431 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 16:54:30.396478 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 16:54:30.397738 6632 raft_consensus.cc:1261] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Refusing update from remote peer a0ed48a38ec84ae0ad749395b8e0d741: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:30.397821 6782 raft_consensus.cc:1261] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Refusing update from remote peer a0ed48a38ec84ae0ad749395b8e0d741: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:30.398028 6864 consensus_queue.cc:1035] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:30.398146 6886 consensus_queue.cc:1035] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:30.398713 6863 tablet_replica.cc:439] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79: TxnStatusTablet state changed. Reason: New leader a0ed48a38ec84ae0ad749395b8e0d741. Latest consensus state: current_term: 1 leader_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } }
I20260619 16:54:30.398799 6863 tablet_replica.cc:442] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:30.398988 6866 tablet_replica.cc:439] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b: TxnStatusTablet state changed. Reason: New leader a0ed48a38ec84ae0ad749395b8e0d741. Latest consensus state: current_term: 1 leader_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } }
I20260619 16:54:30.399080 6866 tablet_replica.cc:442] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:30.399730 6886 tablet_replica.cc:439] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } }
I20260619 16:54:30.399820 6886 tablet_replica.cc:442] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:54:30.400225 6866 tablet_replica.cc:439] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } }
I20260619 16:54:30.400300 6866 tablet_replica.cc:442] T a99753b44da646cf9d604d9670d2fa67 P 9c9c07b39c7642d5bf0282c519f9e49b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:30.400732 6864 tablet_replica.cc:439] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } }
I20260619 16:54:30.400811 6864 tablet_replica.cc:442] T a99753b44da646cf9d604d9670d2fa67 P a0ed48a38ec84ae0ad749395b8e0d741: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:54:30.401077 6863 tablet_replica.cc:439] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } } }
I20260619 16:54:30.401155 6863 tablet_replica.cc:442] T a99753b44da646cf9d604d9670d2fa67 P 1ee47a7f03a747a1abe4df419b0f6d79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:54:30.404556 6854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.402159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49260 (local address 127.30.148.129:38981)
0619 16:54:30.402198 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.402200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.402209 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:54:30.402265 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.402266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.402274 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.402327 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:30.402485 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.403203 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.403887 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.404193 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.404262 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.404362 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:30.404391 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:30.404437 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.404477 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.404493 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:54:30.409379 6854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.406788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:49113 (local address 127.30.148.129:38981)
0619 16:54:30.406824 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.406826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.406966 (+ 140us) server_negotiation.cc:407] Connection header received
0619 16:54:30.406993 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.406994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.407003 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.407073 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:30.407159 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.407878 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.408619 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.408910 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.408978 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.409136 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:30.409164 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:30.409216 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.409260 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.409277 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:54:30.409443 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.406736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:38981 (local address 127.30.148.129:49113)
0619 16:54:30.406897 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:30.406907 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:30.406962 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:30.407074 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:30.407075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:30.407076 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:54:30.407122 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.407123 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.407944 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.407944 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.408543 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.408545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.409000 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.409001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.409050 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.409061 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:54:30.409222 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:54:30.409223 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 16:54:30.409257 (+ 34us) client_negotiation.cc:240] Negotiation successful
0619 16:54:30.409290 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":107,"threads_started":1}
I20260619 16:54:30.411512 6782 raft_consensus.cc:1261] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Refusing update from remote peer a0ed48a38ec84ae0ad749395b8e0d741: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:30.411561 6632 raft_consensus.cc:1261] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Refusing update from remote peer a0ed48a38ec84ae0ad749395b8e0d741: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:30.411736 6864 consensus_queue.cc:1035] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:30.411824 6903 consensus_queue.cc:1035] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:30.426141 6344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:30.426252 6344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:30.426858 6344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666: Sending DeleteTablet for 3 replicas of tablet d2b675655a5e418eba07cca0c5a462c7
I20260619 16:54:30.427143 6622 tablet_service.cc:1514] Processing DeleteTablet for tablet d2b675655a5e418eba07cca0c5a462c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:30 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:45162
I20260619 16:54:30.427232 6880 tablet_replica.cc:330] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79: stopping tablet replica
I20260619 16:54:30.427242 6762 tablet_service.cc:1514] Processing DeleteTablet for tablet d2b675655a5e418eba07cca0c5a462c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:30 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:60382
I20260619 16:54:30.427278 6880 raft_consensus.cc:2229] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:30.427318 6880 raft_consensus.cc:2258] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:30.427325 6879 tablet_replica.cc:330] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b: stopping tablet replica
I20260619 16:54:30.427383 6879 raft_consensus.cc:2229] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:30.427424 6879 raft_consensus.cc:2258] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:30.427767 6482 tablet_service.cc:1514] Processing DeleteTablet for tablet d2b675655a5e418eba07cca0c5a462c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:30 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:49246
I20260619 16:54:30.427848 6878 tablet_replica.cc:330] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741: stopping tablet replica
I20260619 16:54:30.427892 6878 raft_consensus.cc:2229] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:30.427973 6878 raft_consensus.cc:2258] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:30.428167 6880 ts_tablet_manager.cc:1905] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:30.429908 6880 ts_tablet_manager.cc:1918] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:30.429955 6880 log.cc:1196] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/d2b675655a5e418eba07cca0c5a462c7
I20260619 16:54:30.430150 6880 ts_tablet_manager.cc:1939] T d2b675655a5e418eba07cca0c5a462c7 P 1ee47a7f03a747a1abe4df419b0f6d79: Deleting consensus metadata
I20260619 16:54:30.430480 6332 catalog_manager.cc:4841] TS 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130:35635): tablet d2b675655a5e418eba07cca0c5a462c7 (table test-table [id=78a715ddd4e048fb8706a5e2a05edc16]) successfully deleted
I20260619 16:54:30.433701 6879 ts_tablet_manager.cc:1905] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:30.435479 6879 ts_tablet_manager.cc:1918] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:30.435532 6879 log.cc:1196] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/d2b675655a5e418eba07cca0c5a462c7
I20260619 16:54:30.435737 6879 ts_tablet_manager.cc:1939] T d2b675655a5e418eba07cca0c5a462c7 P 9c9c07b39c7642d5bf0282c519f9e49b: Deleting consensus metadata
I20260619 16:54:30.436058 6878 ts_tablet_manager.cc:1905] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:30.436095 6332 catalog_manager.cc:4841] TS 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917): tablet d2b675655a5e418eba07cca0c5a462c7 (table test-table [id=78a715ddd4e048fb8706a5e2a05edc16]) successfully deleted
I20260619 16:54:30.437822 6878 ts_tablet_manager.cc:1918] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:30.437886 6878 log.cc:1196] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/wal/wals/d2b675655a5e418eba07cca0c5a462c7
I20260619 16:54:30.438127 6878 ts_tablet_manager.cc:1939] T d2b675655a5e418eba07cca0c5a462c7 P a0ed48a38ec84ae0ad749395b8e0d741: Deleting consensus metadata
I20260619 16:54:30.438467 6329 catalog_manager.cc:4841] TS a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129:38981): tablet d2b675655a5e418eba07cca0c5a462c7 (table test-table [id=78a715ddd4e048fb8706a5e2a05edc16]) successfully deleted
Jun 19 16:54:30 dist-test-slave-xb9n krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 16:54:30 dist-test-slave-xb9n krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:30 dist-test-slave-xb9n krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 16:54:30.453997 6395 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:30.448196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56750 (local address 127.30.148.190:43241)
0619 16:54:30.448269 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.448271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.448402 (+ 131us) server_negotiation.cc:407] Connection header received
0619 16:54:30.448457 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.448458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.448469 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.448532 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:30.449065 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.449867 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.450575 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.450864 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.450944 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.453935 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56750: BlockingRecv error: recv got EOF from 127.0.0.1:56750 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 16:54:30 dist-test-slave-xb9n krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM
I20260619 16:54:30.461786 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.454752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56752 (local address 127.30.148.190:43241)
0619 16:54:30.454795 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.454797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.454869 (+ 72us) server_negotiation.cc:407] Connection header received
0619 16:54:30.454898 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.454899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.454909 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.454976 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:30.455401 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.456095 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.456747 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.457035 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.457114 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.459115 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:30.459117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:30.459117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:30.459124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:30.460640 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:30.460964 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:30.460965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:30.460966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:30.461038 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:30.461240 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:30.461241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:30.461242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:30.461431 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:30.461540 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.461616 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.461690 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 16:54:30.463259 6344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:30.463426 6344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:30.464829 6482 tablet_service.cc:1467] Processing CreateTablet for tablet 19f77a81ab03466da35fefb500f06b68 (DEFAULT_TABLE table=test-table [id=e198d97644b64b388b47f307b7d34df9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:30.464880 6762 tablet_service.cc:1467] Processing CreateTablet for tablet 19f77a81ab03466da35fefb500f06b68 (DEFAULT_TABLE table=test-table [id=e198d97644b64b388b47f307b7d34df9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:30.464962 6482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f77a81ab03466da35fefb500f06b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:30.465019 6762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f77a81ab03466da35fefb500f06b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:30.466238 6857 tablet_bootstrap.cc:492] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741: Bootstrap starting.
I20260619 16:54:30.466310 6622 tablet_service.cc:1467] Processing CreateTablet for tablet 19f77a81ab03466da35fefb500f06b68 (DEFAULT_TABLE table=test-table [id=e198d97644b64b388b47f307b7d34df9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:30.466425 6622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f77a81ab03466da35fefb500f06b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:30.466446 6856 tablet_bootstrap.cc:492] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b: Bootstrap starting.
I20260619 16:54:30.466656 6857 tablet_bootstrap.cc:654] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:30.466805 6856 tablet_bootstrap.cc:654] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:30.467283 6857 tablet_bootstrap.cc:492] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741: No bootstrap required, opened a new log
I20260619 16:54:30.467336 6857 ts_tablet_manager.cc:1397] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:30.467504 6857 raft_consensus.cc:348] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.467577 6857 raft_consensus.cc:374] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:30.467597 6857 raft_consensus.cc:729] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ed48a38ec84ae0ad749395b8e0d741, State: Initialized, Role: FOLLOWER
I20260619 16:54:30.467626 6857 consensus_queue.cc:260] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.467706 6857 ts_tablet_manager.cc:1428] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:30.467762 6858 tablet_bootstrap.cc:492] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79: Bootstrap starting.
I20260619 16:54:30.468112 6856 tablet_bootstrap.cc:492] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b: No bootstrap required, opened a new log
I20260619 16:54:30.468158 6856 ts_tablet_manager.cc:1397] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:30.468137 6858 tablet_bootstrap.cc:654] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:30.468235 6856 raft_consensus.cc:348] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.468277 6856 raft_consensus.cc:374] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:30.468290 6856 raft_consensus.cc:729] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c9c07b39c7642d5bf0282c519f9e49b, State: Initialized, Role: FOLLOWER
I20260619 16:54:30.468348 6856 consensus_queue.cc:260] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.468441 6856 ts_tablet_manager.cc:1428] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:30.469292 6858 tablet_bootstrap.cc:492] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79: No bootstrap required, opened a new log
I20260619 16:54:30.469331 6858 ts_tablet_manager.cc:1397] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260619 16:54:30.469519 6858 raft_consensus.cc:348] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.469604 6858 raft_consensus.cc:374] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:30.469636 6858 raft_consensus.cc:729] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee47a7f03a747a1abe4df419b0f6d79, State: Initialized, Role: FOLLOWER
I20260619 16:54:30.469686 6858 consensus_queue.cc:260] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.469770 6858 ts_tablet_manager.cc:1428] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:30.546728 6863 raft_consensus.cc:482] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:30.546850 6863 raft_consensus.cc:504] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.547201 6863 leader_election.cc:288] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129:38981), 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917)
I20260619 16:54:30.551055 6854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.547501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:38225 (local address 127.30.148.129:38981)
0619 16:54:30.547546 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.547548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.547630 (+ 82us) server_negotiation.cc:407] Connection header received
0619 16:54:30.547731 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.547732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.547741 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.547792 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:30.548003 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.548702 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.550144 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.550672 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.550760 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.550770 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.550952 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.550993 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:54:30.551067 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.547354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:38981 (local address 127.30.148.130:38225)
0619 16:54:30.547571 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:30.547582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:30.547654 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:30.547833 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:30.547834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:30.547836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:30.547907 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.547909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.548820 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.548820 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.550044 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.550046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.550791 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.550792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.550876 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.550881 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:54:30.550936 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 16:54:30.550974 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":59,"threads_started":1}
I20260619 16:54:30.551086 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.547442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:34345 (local address 127.30.148.131:38917)
0619 16:54:30.547502 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.547504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.547639 (+ 135us) server_negotiation.cc:407] Connection header received
0619 16:54:30.547697 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.547698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.547710 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.547766 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:30.547955 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.548702 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.550232 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.550765 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.550836 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.550842 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.550991 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.551025 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 16:54:30.551301 6502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f77a81ab03466da35fefb500f06b68" candidate_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" is_pre_election: true
I20260619 16:54:30.551411 6502 raft_consensus.cc:2454] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ee47a7f03a747a1abe4df419b0f6d79 in term 0.
I20260619 16:54:30.551538 6927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.547352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:38917 (local address 127.30.148.130:34345)
0619 16:54:30.547551 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:30.547563 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:30.547632 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:30.547782 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:30.547783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:30.547785 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:30.547868 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.547870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.548796 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.548797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.550114 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.550116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.550835 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.550836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.550907 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.550910 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:30.551360 (+ 450us) client_negotiation.cc:240] Negotiation successful
0619 16:54:30.551424 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
I20260619 16:54:30.551584 6573 leader_election.cc:302] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ee47a7f03a747a1abe4df419b0f6d79, a0ed48a38ec84ae0ad749395b8e0d741; no voters:
I20260619 16:54:30.551687 6863 raft_consensus.cc:2781] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:30.551764 6863 raft_consensus.cc:482] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:30.551786 6863 raft_consensus.cc:3037] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.551930 6782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f77a81ab03466da35fefb500f06b68" candidate_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" is_pre_election: true
I20260619 16:54:30.552030 6782 raft_consensus.cc:2454] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ee47a7f03a747a1abe4df419b0f6d79 in term 0.
I20260619 16:54:30.552619 6863 raft_consensus.cc:504] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.552759 6863 leader_election.cc:288] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [CANDIDATE]: Term 1 election: Requested vote from peers a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129:38981), 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917)
I20260619 16:54:30.552886 6782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f77a81ab03466da35fefb500f06b68" candidate_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c9c07b39c7642d5bf0282c519f9e49b"
I20260619 16:54:30.552922 6502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f77a81ab03466da35fefb500f06b68" candidate_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ed48a38ec84ae0ad749395b8e0d741"
I20260619 16:54:30.552959 6782 raft_consensus.cc:3037] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.552989 6502 raft_consensus.cc:3037] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.553692 6502 raft_consensus.cc:2454] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ee47a7f03a747a1abe4df419b0f6d79 in term 1.
I20260619 16:54:30.553791 6782 raft_consensus.cc:2454] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ee47a7f03a747a1abe4df419b0f6d79 in term 1.
I20260619 16:54:30.553879 6573 leader_election.cc:302] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ee47a7f03a747a1abe4df419b0f6d79, a0ed48a38ec84ae0ad749395b8e0d741; no voters:
I20260619 16:54:30.553954 6863 raft_consensus.cc:2781] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:30.554054 6863 raft_consensus.cc:686] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 LEADER]: Becoming Leader. State: Replica: 1ee47a7f03a747a1abe4df419b0f6d79, State: Running, Role: LEADER
I20260619 16:54:30.554121 6863 consensus_queue.cc:237] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.554816 6343 catalog_manager.cc:5495] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130). New cstate: current_term: 1 leader_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } health_report { overall_health: UNKNOWN } } }
I20260619 16:54:30.562847 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.560265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.30.148.130:35635)
0619 16:54:30.560303 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.560305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.560315 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:30.560345 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.560345 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.560363 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.560458 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:30.560571 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.561278 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.562043 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.562333 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.562403 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.562539 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:30.562654 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:30.562728 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.562765 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.562784 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:54:30.563570 6502 raft_consensus.cc:1261] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Refusing update from remote peer 1ee47a7f03a747a1abe4df419b0f6d79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:30.563635 6782 raft_consensus.cc:1261] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Refusing update from remote peer 1ee47a7f03a747a1abe4df419b0f6d79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:30.563831 6928 consensus_queue.cc:1035] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:30.563930 6863 consensus_queue.cc:1035] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:30.567291 6344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:30.567363 6344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:30.567988 6344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666: Sending DeleteTablet for 3 replicas of tablet 19f77a81ab03466da35fefb500f06b68
I20260619 16:54:30.568257 6482 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f77a81ab03466da35fefb500f06b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:30 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:49246
I20260619 16:54:30.568346 6762 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f77a81ab03466da35fefb500f06b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:30 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:60382
I20260619 16:54:30.568425 6878 tablet_replica.cc:330] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741: stopping tablet replica
I20260619 16:54:30.568477 6878 raft_consensus.cc:2229] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:30.568521 6878 raft_consensus.cc:2258] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:30.568578 6879 tablet_replica.cc:330] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b: stopping tablet replica
I20260619 16:54:30.568523 6344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 16:54:30.568630 6879 raft_consensus.cc:2229] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Raft consensus shutting down.
W20260619 16:54:30.568636 6344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:30.568665 6879 raft_consensus.cc:2258] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:30.568856 6622 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f77a81ab03466da35fefb500f06b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:30 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:45162
I20260619 16:54:30.568928 6880 tablet_replica.cc:330] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79: stopping tablet replica
I20260619 16:54:30.568980 6880 raft_consensus.cc:2229] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:30.569056 6880 raft_consensus.cc:2258] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:30.569375 6878 ts_tablet_manager.cc:1905] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:30.569496 6880 ts_tablet_manager.cc:1905] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:30.570809 6878 ts_tablet_manager.cc:1918] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:30.570858 6878 log.cc:1196] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-0/wal/wals/19f77a81ab03466da35fefb500f06b68
I20260619 16:54:30.570875 6880 ts_tablet_manager.cc:1918] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:30.570924 6880 log.cc:1196] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/19f77a81ab03466da35fefb500f06b68
I20260619 16:54:30.571074 6878 ts_tablet_manager.cc:1939] T 19f77a81ab03466da35fefb500f06b68 P a0ed48a38ec84ae0ad749395b8e0d741: Deleting consensus metadata
I20260619 16:54:30.571128 6880 ts_tablet_manager.cc:1939] T 19f77a81ab03466da35fefb500f06b68 P 1ee47a7f03a747a1abe4df419b0f6d79: Deleting consensus metadata
I20260619 16:54:30.571383 6332 catalog_manager.cc:4841] TS 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130:35635): tablet 19f77a81ab03466da35fefb500f06b68 (table test-table [id=e198d97644b64b388b47f307b7d34df9]) successfully deleted
I20260619 16:54:30.571383 6329 catalog_manager.cc:4841] TS a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129:38981): tablet 19f77a81ab03466da35fefb500f06b68 (table test-table [id=e198d97644b64b388b47f307b7d34df9]) successfully deleted
I20260619 16:54:30.571799 6879 ts_tablet_manager.cc:1905] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:30.572479 6762 tablet_service.cc:1467] Processing CreateTablet for tablet 8aefc4a28d764d55a5b9e62cb9d20abc (DEFAULT_TABLE table=test-table [id=0501ab20a52d424d82c48eb5c095319e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:30.572577 6762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aefc4a28d764d55a5b9e62cb9d20abc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:30.573202 6879 ts_tablet_manager.cc:1918] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:30.573251 6879 log.cc:1196] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/19f77a81ab03466da35fefb500f06b68
I20260619 16:54:30.573493 6879 ts_tablet_manager.cc:1939] T 19f77a81ab03466da35fefb500f06b68 P 9c9c07b39c7642d5bf0282c519f9e49b: Deleting consensus metadata
I20260619 16:54:30.573772 6332 catalog_manager.cc:4841] TS 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917): tablet 19f77a81ab03466da35fefb500f06b68 (table test-table [id=e198d97644b64b388b47f307b7d34df9]) successfully deleted
I20260619 16:54:30.573828 6856 tablet_bootstrap.cc:492] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b: Bootstrap starting.
I20260619 16:54:30.573909 6622 tablet_service.cc:1467] Processing CreateTablet for tablet 8aefc4a28d764d55a5b9e62cb9d20abc (DEFAULT_TABLE table=test-table [id=0501ab20a52d424d82c48eb5c095319e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:30.574003 6622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aefc4a28d764d55a5b9e62cb9d20abc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:30.574106 6856 tablet_bootstrap.cc:654] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:30.574672 6856 tablet_bootstrap.cc:492] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b: No bootstrap required, opened a new log
I20260619 16:54:30.574713 6856 ts_tablet_manager.cc:1397] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:30.574802 6856 raft_consensus.cc:348] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.574855 6856 raft_consensus.cc:374] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:30.574873 6856 raft_consensus.cc:729] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c9c07b39c7642d5bf0282c519f9e49b, State: Initialized, Role: FOLLOWER
I20260619 16:54:30.574921 6856 consensus_queue.cc:260] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.575017 6856 ts_tablet_manager.cc:1428] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:30.575150 6858 tablet_bootstrap.cc:492] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79: Bootstrap starting.
I20260619 16:54:30.575505 6858 tablet_bootstrap.cc:654] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:30.576134 6482 tablet_service.cc:1467] Processing CreateTablet for tablet 8aefc4a28d764d55a5b9e62cb9d20abc (DEFAULT_TABLE table=test-table [id=0501ab20a52d424d82c48eb5c095319e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:30.576254 6482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aefc4a28d764d55a5b9e62cb9d20abc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:30.576877 6858 tablet_bootstrap.cc:492] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79: No bootstrap required, opened a new log
I20260619 16:54:30.576924 6858 ts_tablet_manager.cc:1397] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:54:30.577085 6858 raft_consensus.cc:348] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.577142 6858 raft_consensus.cc:374] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:30.577163 6858 raft_consensus.cc:729] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee47a7f03a747a1abe4df419b0f6d79, State: Initialized, Role: FOLLOWER
I20260619 16:54:30.577203 6858 consensus_queue.cc:260] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.577288 6858 ts_tablet_manager.cc:1428] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:30.577414 6857 tablet_bootstrap.cc:492] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741: Bootstrap starting.
I20260619 16:54:30.577700 6857 tablet_bootstrap.cc:654] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:30.578804 6857 tablet_bootstrap.cc:492] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741: No bootstrap required, opened a new log
I20260619 16:54:30.578846 6857 ts_tablet_manager.cc:1397] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:30.578941 6857 raft_consensus.cc:348] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.579000 6857 raft_consensus.cc:374] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:30.579036 6857 raft_consensus.cc:729] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ed48a38ec84ae0ad749395b8e0d741, State: Initialized, Role: FOLLOWER
I20260619 16:54:30.579079 6857 consensus_queue.cc:260] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.579164 6857 ts_tablet_manager.cc:1428] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:30.679850 6933 raft_consensus.cc:482] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:30.679944 6933 raft_consensus.cc:504] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.680116 6933 leader_election.cc:288] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129:38981), 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917)
I20260619 16:54:30.680297 6502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aefc4a28d764d55a5b9e62cb9d20abc" candidate_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" is_pre_election: true
I20260619 16:54:30.680334 6782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aefc4a28d764d55a5b9e62cb9d20abc" candidate_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" is_pre_election: true
I20260619 16:54:30.680399 6502 raft_consensus.cc:2454] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ee47a7f03a747a1abe4df419b0f6d79 in term 0.
I20260619 16:54:30.680409 6782 raft_consensus.cc:2454] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ee47a7f03a747a1abe4df419b0f6d79 in term 0.
I20260619 16:54:30.680542 6576 leader_election.cc:302] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ee47a7f03a747a1abe4df419b0f6d79, 9c9c07b39c7642d5bf0282c519f9e49b; no voters:
I20260619 16:54:30.680641 6933 raft_consensus.cc:2781] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:30.680675 6933 raft_consensus.cc:482] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:30.680693 6933 raft_consensus.cc:3037] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.681492 6933 raft_consensus.cc:504] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.681610 6933 leader_election.cc:288] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [CANDIDATE]: Term 1 election: Requested vote from peers a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129:38981), 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917)
I20260619 16:54:30.681767 6782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aefc4a28d764d55a5b9e62cb9d20abc" candidate_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c9c07b39c7642d5bf0282c519f9e49b"
I20260619 16:54:30.681766 6502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aefc4a28d764d55a5b9e62cb9d20abc" candidate_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ed48a38ec84ae0ad749395b8e0d741"
I20260619 16:54:30.681840 6502 raft_consensus.cc:3037] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.681840 6782 raft_consensus.cc:3037] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:30.682586 6782 raft_consensus.cc:2454] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ee47a7f03a747a1abe4df419b0f6d79 in term 1.
I20260619 16:54:30.682585 6502 raft_consensus.cc:2454] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ee47a7f03a747a1abe4df419b0f6d79 in term 1.
I20260619 16:54:30.682745 6576 leader_election.cc:302] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ee47a7f03a747a1abe4df419b0f6d79, 9c9c07b39c7642d5bf0282c519f9e49b; no voters:
I20260619 16:54:30.682845 6933 raft_consensus.cc:2781] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:30.682889 6933 raft_consensus.cc:686] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 LEADER]: Becoming Leader. State: Replica: 1ee47a7f03a747a1abe4df419b0f6d79, State: Running, Role: LEADER
I20260619 16:54:30.682937 6933 consensus_queue.cc:237] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } }
I20260619 16:54:30.683497 6344 catalog_manager.cc:5495] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130). New cstate: current_term: 1 leader_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ee47a7f03a747a1abe4df419b0f6d79" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 } health_report { overall_health: UNKNOWN } } }
I20260619 16:54:30.700322 6854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.697975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:46557 (local address 127.30.148.129:38981)
0619 16:54:30.698045 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.698047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.698125 (+ 78us) server_negotiation.cc:407] Connection header received
0619 16:54:30.698190 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.698191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.698201 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.698281 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:30.698396 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.698866 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.699619 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.699828 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.699904 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.700062 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:30.700097 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:30.700162 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.700232 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.700253 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 16:54:30.700333 6941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.697896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:38981 (local address 127.30.148.130:46557)
0619 16:54:30.698069 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:30.698080 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:30.698139 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:30.698291 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:30.698291 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:30.698293 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:54:30.698340 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.698342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.698944 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.698945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.699521 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.699522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.699901 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.699902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.699956 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.699970 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:54:30.700176 (+ 206us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:54:30.700178 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:54:30.700215 (+ 37us) client_negotiation.cc:240] Negotiation successful
0619 16:54:30.700261 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260619 16:54:30.702070 6782 raft_consensus.cc:1261] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Refusing update from remote peer 1ee47a7f03a747a1abe4df419b0f6d79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:30.702081 6502 raft_consensus.cc:1261] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Refusing update from remote peer 1ee47a7f03a747a1abe4df419b0f6d79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:30.702297 6933 consensus_queue.cc:1035] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ed48a38ec84ae0ad749395b8e0d741" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:30.702402 6928 consensus_queue.cc:1035] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c9c07b39c7642d5bf0282c519f9e49b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 38917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:30.708734 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.706317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:39915 (local address 127.30.148.130:35635)
0619 16:54:30.706451 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:30.706453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:30.706462 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:54:30.706480 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:30.706481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:30.706490 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:30.706568 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:30.706732 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.707316 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.708001 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.708233 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.708286 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.708464 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:30.708496 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:30.708551 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:30.708634 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 16:54:30.708656 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260619 16:54:30.708735 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.706256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:35635 (local address 127.30.148.129:39915)
0619 16:54:30.706312 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:30.706324 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:30.706364 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:30.706589 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:30.706590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:30.706591 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:54:30.706633 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.706634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.707381 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.707382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.707935 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:30.707937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:30.708297 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:30.708298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:30.708365 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:30.708371 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:54:30.708567 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:54:30.708569 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:54:30.708618 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 16:54:30.708664 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 16:54:30.714831 6344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:30.714907 6344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:30.715466 6344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 982afb07b9144b649097ef27ce719666: Sending DeleteTablet for 3 replicas of tablet 8aefc4a28d764d55a5b9e62cb9d20abc
I20260619 16:54:30.715715 6622 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aefc4a28d764d55a5b9e62cb9d20abc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:30 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:45162
I20260619 16:54:30.715729 6482 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aefc4a28d764d55a5b9e62cb9d20abc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:30 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:49246
I20260619 16:54:30.715749 6762 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aefc4a28d764d55a5b9e62cb9d20abc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:30 UTC) from {username='oryx', principal='oryx/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:60382
I20260619 16:54:30.715808 6880 tablet_replica.cc:330] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79: stopping tablet replica
I20260619 16:54:30.715833 6879 tablet_replica.cc:330] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b: stopping tablet replica
I20260619 16:54:30.715855 6880 raft_consensus.cc:2229] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:54:30.715888 6879 raft_consensus.cc:2229] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:30.715906 6880 raft_consensus.cc:2258] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:30.715925 6879 raft_consensus.cc:2258] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:30.715994 6878 tablet_replica.cc:330] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741: stopping tablet replica
I20260619 16:54:30.716056 6878 raft_consensus.cc:2229] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:30.716094 6878 raft_consensus.cc:2258] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:30.716257 6879 ts_tablet_manager.cc:1905] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:30.716367 6880 ts_tablet_manager.cc:1905] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:30.716450 6878 ts_tablet_manager.cc:1905] T 8aefc4a28d764d55a5b9e62cb9d20abc P a0ed48a38ec84ae0ad749395b8e0d741: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:30.716507 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 6417
I20260619 16:54:30.717962 6880 ts_tablet_manager.cc:1918] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:30.718011 6880 log.cc:1196] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/8aefc4a28d764d55a5b9e62cb9d20abc
I20260619 16:54:30.718261 6880 ts_tablet_manager.cc:1939] T 8aefc4a28d764d55a5b9e62cb9d20abc P 1ee47a7f03a747a1abe4df419b0f6d79: Deleting consensus metadata
I20260619 16:54:30.718562 6332 catalog_manager.cc:4841] TS 1ee47a7f03a747a1abe4df419b0f6d79 (127.30.148.130:35635): tablet 8aefc4a28d764d55a5b9e62cb9d20abc (table test-table [id=0501ab20a52d424d82c48eb5c095319e]) successfully deleted
I20260619 16:54:30.719503 6879 ts_tablet_manager.cc:1918] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:54:30.719558 6879 log.cc:1196] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/8aefc4a28d764d55a5b9e62cb9d20abc
I20260619 16:54:30.719761 6879 ts_tablet_manager.cc:1939] T 8aefc4a28d764d55a5b9e62cb9d20abc P 9c9c07b39c7642d5bf0282c519f9e49b: Deleting consensus metadata
I20260619 16:54:30.720084 6332 catalog_manager.cc:4841] TS 9c9c07b39c7642d5bf0282c519f9e49b (127.30.148.131:38917): tablet 8aefc4a28d764d55a5b9e62cb9d20abc (table test-table [id=0501ab20a52d424d82c48eb5c095319e]) successfully deleted
W20260619 16:54:30.724380 6329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:38981 (error 108)
I20260619 16:54:30.724646 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 6557
I20260619 16:54:30.724859 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:30.724690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:38981 (local address 127.0.0.1:49274)
0619 16:54:30.724782 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:30.724820 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":56,"threads_started":1}
W20260619 16:54:30.725405 6329 catalog_manager.cc:4584] TS a0ed48a38ec84ae0ad749395b8e0d741 (127.30.148.129:38981): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8aefc4a28d764d55a5b9e62cb9d20abc: Network error: Client connection negotiation failed: client connection to 127.30.148.129:38981: connect: Connection refused (error 111)
I20260619 16:54:30.730871 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 6697
I20260619 16:54:30.736124 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 6313
2026-06-19T16:54:30Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2571 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:30 dist-test-slave-xb9n krb5kdc[6948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:30 dist-test-slave-xb9n krb5kdc[6948](info): set up 2 sockets
Jun 19 16:54:30 dist-test-slave-xb9n krb5kdc[6948](info): commencing operation
krb5kdc: starting...
W20260619 16:54:31.769796 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:31 dist-test-slave-xb9n krb5kdc[6948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:54:31.878309 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36095
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:36095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:31.955144 6961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:31.955333 6961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:31.955354 6961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:31.956466 6961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:31.956506 6961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:31.956521 6961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:31.956568 6961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:31.956597 6961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:31.958034 6961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:36095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:31.958266 6961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:31.958499 6961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:31.963052 6966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:31.963095 6967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:31.963083 6969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:31.963121 6961 server_base.cc:1034] running on GCE node
I20260619 16:54:31.963425 6961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:31.963711 6961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:31.964845 6961 hybrid_clock.cc:648] HybridClock initialized: now 1781888071964826 us; error 38 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T16:54:31Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1228 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:32 dist-test-slave-xb9n krb5kdc[6974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:32 dist-test-slave-xb9n krb5kdc[6974](info): set up 2 sockets
Jun 19 16:54:32 dist-test-slave-xb9n krb5kdc[6974](info): commencing operation
krb5kdc: starting...
W20260619 16:54:32.997861 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:33 dist-test-slave-xb9n krb5kdc[6974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:54:33.111555 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:32943
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:32943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:33.189685 6987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:33.189877 6987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:33.189898 6987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:33.190953 6987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:33.190989 6987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:33.191001 6987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:33.191015 6987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:33.191028 6987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:33.192409 6987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:32943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:32943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:33.192605 6987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:33.192780 6987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:33.197696 6992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:33.197719 6987 server_base.cc:1034] running on GCE node
W20260619 16:54:33.197686 6993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:33.197686 6995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:33.198213 6987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:33.198510 6987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:33.199698 6987 hybrid_clock.cc:648] HybridClock initialized: now 1781888073199690 us; error 35 us; skew 500 ppm
Jun 19 16:54:33 dist-test-slave-xb9n krb5kdc[6974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888073, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:33.202105 6987 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:54:33.202549 6987 webserver.cc:460] Webserver started at http://127.30.148.190:39843/ using document root <none> and password file <none>
I20260619 16:54:33.202773 6987 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:33.202838 6987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:33.202983 6987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:33.203990 6987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "ee48569a155a41d0b5483c3a1686e588"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "7b7d6c4908991d6d726168af414b6ef1"
server_key_iv: "c4ad3f69a35c1a2c6e8193b79e3d4ad0"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.204355 6987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "ee48569a155a41d0b5483c3a1686e588"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "7b7d6c4908991d6d726168af414b6ef1"
server_key_iv: "c4ad3f69a35c1a2c6e8193b79e3d4ad0"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.205674 6987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:54:33.206415 7002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:33.206590 6987 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:33.206686 6987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "ee48569a155a41d0b5483c3a1686e588"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "7b7d6c4908991d6d726168af414b6ef1"
server_key_iv: "c4ad3f69a35c1a2c6e8193b79e3d4ad0"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.206761 6987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:33.222450 6987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:33.224206 6987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:33.224359 6987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:33.228408 6987 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:32943
I20260619 16:54:33.228509 7054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:32943 every 8 connection(s)
I20260619 16:54:33.228808 6987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:33.229691 7055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:33.231853 7055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588: Bootstrap starting.
I20260619 16:54:33.232422 7055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:33.232769 7055 log.cc:824] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:33.238075 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 6987
I20260619 16:54:33.238180 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:33.238291 7055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588: No bootstrap required, opened a new log
I20260619 16:54:33.238376 31314 external_mini_cluster.cc:1428] Setting key 5157466322b33747584b42856b6144db
I20260619 16:54:33.240456 7055 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48569a155a41d0b5483c3a1686e588" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32943 } }
I20260619 16:54:33.240619 7055 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:33.240657 7055 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee48569a155a41d0b5483c3a1686e588, State: Initialized, Role: FOLLOWER
I20260619 16:54:33.240826 7055 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48569a155a41d0b5483c3a1686e588" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32943 } }
I20260619 16:54:33.240935 7055 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:33.240969 7055 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:33.241015 7055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:33.241736 7055 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48569a155a41d0b5483c3a1686e588" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32943 } }
I20260619 16:54:33.241824 7055 leader_election.cc:302] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee48569a155a41d0b5483c3a1686e588; no voters:
I20260619 16:54:33.242055 7055 leader_election.cc:288] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:33.242081 7060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:33.242230 7060 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 1 LEADER]: Becoming Leader. State: Replica: ee48569a155a41d0b5483c3a1686e588, State: Running, Role: LEADER
I20260619 16:54:33.242311 7060 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48569a155a41d0b5483c3a1686e588" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32943 } }
I20260619 16:54:33.242368 7055 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:33.242635 7060 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee48569a155a41d0b5483c3a1686e588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48569a155a41d0b5483c3a1686e588" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32943 } } }
I20260619 16:54:33.242815 7060 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:33.243005 7060 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee48569a155a41d0b5483c3a1686e588. Latest consensus state: current_term: 1 leader_uuid: "ee48569a155a41d0b5483c3a1686e588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48569a155a41d0b5483c3a1686e588" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32943 } } }
I20260619 16:54:33.243076 7060 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:33.243346 7068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:33.243852 7068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:33.245782 7068 catalog_manager.cc:1347] Generated new cluster ID: f153ab8df0a34f5388385877e7a42375
I20260619 16:54:33.245841 7068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:54:33 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:33.254222 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:33.238878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50250 (local address 127.30.148.190:32943)
0619 16:54:33.239115 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:33.239124 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:33.239760 (+ 636us) server_negotiation.cc:407] Connection header received
0619 16:54:33.240432 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:33.240439 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:33.240502 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:33.240639 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:33.241467 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.243092 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.244249 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.244901 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.245016 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:33.250387 (+ 5371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:33.250391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:33.250391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:33.250403 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:33.252607 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:33.253000 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:33.253003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:33.253003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:33.253086 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:33.253294 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:33.253295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:33.253295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:33.253605 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:33.253791 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:33.253858 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 16:54:33.254017 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":101,"threads_started":1}
I20260619 16:54:33.259503 7068 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:33.260176 7068 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:33.272159 7068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588: Generated new TSK 0
I20260619 16:54:33.272331 7068 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:54:33.296000 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:32943
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:33.374909 7081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:33.375144 7081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:33.375182 7081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:33.376327 7081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:33.376391 7081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:33.376437 7081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:33.377883 7081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:32943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:33.378146 7081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:33.378363 7081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:54:33.383258 7081 server_base.cc:1034] running on GCE node
W20260619 16:54:33.383194 7087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:33.383203 7089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:33.383212 7086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:33.383610 7081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:33.383849 7081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:33.385008 7081 hybrid_clock.cc:648] HybridClock initialized: now 1781888073385000 us; error 29 us; skew 500 ppm
Jun 19 16:54:33 dist-test-slave-xb9n krb5kdc[6974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888073, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:33.387476 7081 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:54:33.387905 7081 webserver.cc:460] Webserver started at http://127.30.148.129:41285/ using document root <none> and password file <none>
I20260619 16:54:33.388147 7081 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:33.388216 7081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:33.388362 7081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:33.389452 7081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "66bb6790154f43db97b53e91dc8b36fb"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "9689e050a7bab381f862a360a92f755e"
server_key_iv: "02204be1fd6ed849ef9e662e18ddd836"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.389809 7081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "66bb6790154f43db97b53e91dc8b36fb"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "9689e050a7bab381f862a360a92f755e"
server_key_iv: "02204be1fd6ed849ef9e662e18ddd836"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.391113 7081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:33.391803 7096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:33.391968 7081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:33.392076 7081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "66bb6790154f43db97b53e91dc8b36fb"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "9689e050a7bab381f862a360a92f755e"
server_key_iv: "02204be1fd6ed849ef9e662e18ddd836"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.392148 7081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:33.418717 7081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:33.420642 7081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:33.420811 7081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:33.421056 7081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:33.421392 7081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:33.421448 7081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:33.421489 7081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:33.421516 7081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:33.427054 7081 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:41243
I20260619 16:54:33.427099 7209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:41243 every 8 connection(s)
I20260619 16:54:33.427404 7081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:33.430723 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 7081
I20260619 16:54:33.430817 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:33.431010 31314 external_mini_cluster.cc:1428] Setting key bca3ca7a8d9099abd248894a83055f74
Jun 19 16:54:33 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888073, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:33.437565 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:33.428139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:43925 (local address 127.30.148.190:32943)
0619 16:54:33.428205 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:33.428207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:33.428411 (+ 204us) server_negotiation.cc:407] Connection header received
0619 16:54:33.428947 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:33.428948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:33.428962 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:33.429015 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:33.430031 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.430571 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.431890 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.432225 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.432303 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:33.434674 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:33.434675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:33.434675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:33.434684 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:33.436124 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:33.436547 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:33.436549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:33.436549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:33.436614 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:33.436868 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:33.436869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:33.436869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:33.436987 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:33.437071 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:33.437343 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 16:54:33.437455 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 16:54:33.437729 7212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:33.428112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:43925)
0619 16:54:33.428293 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:33.428327 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:33.428837 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:33.429082 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:33.429093 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:33.429724 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:33.429913 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:33.429918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.430673 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.430674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:33.431812 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:33.431815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.432315 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.432315 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:33.432418 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:33.432994 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:33.433005 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:33.434576 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:33.436228 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:33.436235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:33.436238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:33.436461 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:33.436681 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:33.436682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:33.436683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:33.436796 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:33.437060 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:33.437066 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:33.437207 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 16:54:33.437321 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 16:54:33.437496 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":90,"threads_started":1}
I20260619 16:54:33.438198 7210 heartbeater.cc:347] Connected to a master server at 127.30.148.190:32943
I20260619 16:54:33.438294 7210 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:33.438467 7210 heartbeater.cc:511] Master 127.30.148.190:32943 requested a full tablet report, sending...
I20260619 16:54:33.438997 7019 ts_manager.cc:194] Registered new tserver with Master: 66bb6790154f43db97b53e91dc8b36fb (127.30.148.129:41243)
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
I20260619 16:54:33.439672 7019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:43925
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:54:33.455395 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:32943
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:33.535447 7215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:33.535634 7215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:33.535657 7215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:33.536765 7215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:33.536801 7215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:33.536826 7215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:33.538215 7215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:32943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:33.538395 7215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:33.538573 7215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:33.543437 7223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:33.543437 7220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:33.543520 7221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:33.543774 7215 server_base.cc:1034] running on GCE node
I20260619 16:54:33.543978 7215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:33.544225 7215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:33.545414 7215 hybrid_clock.cc:648] HybridClock initialized: now 1781888073545400 us; error 24 us; skew 500 ppm
Jun 19 16:54:33 dist-test-slave-xb9n krb5kdc[6974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888073, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:33.547791 7215 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:54:33.548213 7215 webserver.cc:460] Webserver started at http://127.30.148.130:42007/ using document root <none> and password file <none>
I20260619 16:54:33.548452 7215 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:33.548520 7215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:33.548684 7215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:33.549772 7215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "38c89d42433d422596f8e52c903734a1"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "2ef13afd873d77d223f7908138084a64"
server_key_iv: "76814fae55a61f11a5fc3feb6515baba"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.550129 7215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "38c89d42433d422596f8e52c903734a1"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "2ef13afd873d77d223f7908138084a64"
server_key_iv: "76814fae55a61f11a5fc3feb6515baba"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.551386 7215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:33.552086 7230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:33.552254 7215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:33.552349 7215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "38c89d42433d422596f8e52c903734a1"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "2ef13afd873d77d223f7908138084a64"
server_key_iv: "76814fae55a61f11a5fc3feb6515baba"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.552429 7215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:33.580554 7215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:33.582383 7215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:33.582556 7215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:33.582773 7215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:33.583078 7215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:33.583132 7215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:33.583164 7215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:33.583191 7215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:33.589169 7215 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:37837
I20260619 16:54:33.589212 7343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:37837 every 8 connection(s)
I20260619 16:54:33.589589 7215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:54:33.590058 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 7215
I20260619 16:54:33.590180 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:33.590401 31314 external_mini_cluster.cc:1428] Setting key 04db10d7ad175df809ddbaab1222604e
Jun 19 16:54:33 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888073, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
I20260619 16:54:33.599881 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:33.590326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:45643 (local address 127.30.148.190:32943)
0619 16:54:33.590391 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:33.590393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:33.590627 (+ 234us) server_negotiation.cc:407] Connection header received
0619 16:54:33.591281 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:33.591282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:33.591295 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:33.591341 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:33.592265 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.592772 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.593878 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.594188 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.594263 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:33.597001 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:33.597002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:33.597002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:33.597009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:33.598395 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:33.598882 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:33.598883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:33.598883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:33.598933 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:33.599223 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:33.599223 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:33.599224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:33.599321 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:33.599367 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:33.599687 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 16:54:33.599786 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 16:54:33.600118 7346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:33.590321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:45643)
0619 16:54:33.590498 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:33.590538 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:33.591172 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:33.591408 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:33.591412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:33.591976 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:33.592159 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:33.592163 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.592866 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.592867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:33.593779 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:33.593781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.594284 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.594286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:33.594424 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:33.594976 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:33.594984 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:33.596904 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:33.598485 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:33.598491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:33.598493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:33.598803 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:33.598999 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:33.599000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:33.599001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:33.599146 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:33.599388 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:33.599395 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:33.599555 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 16:54:33.599671 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 16:54:33.599857 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260619 16:54:33.600600 7344 heartbeater.cc:347] Connected to a master server at 127.30.148.190:32943
I20260619 16:54:33.600705 7344 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:33.600873 7344 heartbeater.cc:511] Master 127.30.148.190:32943 requested a full tablet report, sending...
I20260619 16:54:33.601326 7019 ts_manager.cc:194] Registered new tserver with Master: 38c89d42433d422596f8e52c903734a1 (127.30.148.130:37837)
I20260619 16:54:33.601714 7019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:45643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:54:33.615679 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:32943
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:33.693506 7349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:33.693732 7349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:33.693768 7349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:33.694945 7349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:33.695019 7349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:33.695081 7349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:33.696436 7349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:32943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:33.696609 7349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:33.696789 7349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:33.701530 7355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:33.701562 7354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:33.701540 7357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:33.701570 7349 server_base.cc:1034] running on GCE node
I20260619 16:54:33.701992 7349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:33.702239 7349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:33.703397 7349 hybrid_clock.cc:648] HybridClock initialized: now 1781888073703391 us; error 36 us; skew 500 ppm
Jun 19 16:54:33 dist-test-slave-xb9n krb5kdc[6974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888073, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:33.705919 7349 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:54:33.706284 7349 webserver.cc:460] Webserver started at http://127.30.148.131:42981/ using document root <none> and password file <none>
I20260619 16:54:33.706519 7349 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:33.706564 7349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:33.706709 7349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:33.707723 7349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "ba5ac18b2fbf4004ac5861a41f0bd2f1"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "8c5f13f2d7f6cace21403855f2d1db76"
server_key_iv: "d5a2955f69667195edf9480fd455a1f8"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.708081 7349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "ba5ac18b2fbf4004ac5861a41f0bd2f1"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "8c5f13f2d7f6cace21403855f2d1db76"
server_key_iv: "d5a2955f69667195edf9480fd455a1f8"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.709385 7349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 16:54:33.710156 7364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:33.710323 7349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:33.710424 7349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "ba5ac18b2fbf4004ac5861a41f0bd2f1"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "8c5f13f2d7f6cace21403855f2d1db76"
server_key_iv: "d5a2955f69667195edf9480fd455a1f8"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.710495 7349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:33.723699 7349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:33.725528 7349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:33.725634 7349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:33.725792 7349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:33.726083 7349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:33.726116 7349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:33.726168 7349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:33.726205 7349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:33.731757 7349 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:42081
I20260619 16:54:33.731833 7477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:42081 every 8 connection(s)
I20260619 16:54:33.732134 7349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:54:33 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888073, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:33.739820 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 7349
I20260619 16:54:33.739931 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:33.740127 31314 external_mini_cluster.cc:1428] Setting key a67539d8fddce0e40b6a127fd8fbf15c
I20260619 16:54:33.740995 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:33.732827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:54871 (local address 127.30.148.190:32943)
0619 16:54:33.732888 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:33.732890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:33.733099 (+ 209us) server_negotiation.cc:407] Connection header received
0619 16:54:33.733560 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:33.733561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:33.733571 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:33.733617 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:33.734465 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.734949 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.735867 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.736042 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.736092 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:33.738181 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:33.738182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:33.738183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:33.738188 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:33.739345 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:33.740011 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:33.740012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:33.740012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:33.740073 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:33.740367 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:33.740368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:33.740369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:33.740490 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:33.740565 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:33.740826 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 16:54:33.740908 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 16:54:33.741179 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:33.732814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:54871)
0619 16:54:33.732983 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:33.733015 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:33.733482 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:33.733674 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:33.733677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:33.734215 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:33.734393 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:33.734398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.735027 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.735027 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:33.735798 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:33.735799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:33.736111 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:33.736111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:33.736212 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:33.736720 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:33.736728 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:33.738105 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:33.739414 (+ 1309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:33.739419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:33.739421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:33.739620 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:33.740157 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:33.740158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:33.740158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:33.740297 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:33.740566 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:33.740573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:33.740695 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 16:54:33.740802 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 16:54:33.740950 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
I20260619 16:54:33.741660 7478 heartbeater.cc:347] Connected to a master server at 127.30.148.190:32943
I20260619 16:54:33.741739 7478 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:33.741963 7478 heartbeater.cc:511] Master 127.30.148.190:32943 requested a full tablet report, sending...
I20260619 16:54:33.742313 7019 ts_manager.cc:194] Registered new tserver with Master: ba5ac18b2fbf4004ac5861a41f0bd2f1 (127.30.148.131:42081)
I20260619 16:54:33.742694 7019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:54871
I20260619 16:54:33.751775 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab.
Entry for principal oryx/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab.
WARNING: no policy specified for oryx/127.30.148.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab.
Entry for principal oryx/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab.
WARNING: no policy specified for oryx/127.30.148.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab.
Entry for principal oryx/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab.
I20260619 16:54:33.815111 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 7081
I20260619 16:54:33.819521 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 7215
I20260619 16:54:33.823498 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 7349
I20260619 16:54:33.827555 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 6987
I20260619 16:54:33.832508 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:32943
--webserver_interface=127.30.148.190
--webserver_port=39843
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:32943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:33.910198 7487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:33.910413 7487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:33.910449 7487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:33.911566 7487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:33.911633 7487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:33.911655 7487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:33.911677 7487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:33.911706 7487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:33.913127 7487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:32943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:32943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=39843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:33.913466 7487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:33.913689 7487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:33.918376 7495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:33.918380 7492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:33.918464 7487 server_base.cc:1034] running on GCE node
W20260619 16:54:33.918756 7493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:33.919023 7487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:33.919365 7487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:33.920519 7487 hybrid_clock.cc:648] HybridClock initialized: now 1781888073920511 us; error 27 us; skew 500 ppm
Jun 19 16:54:33 dist-test-slave-xb9n krb5kdc[6974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888073, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:33.923099 7487 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:54:33.923573 7487 webserver.cc:460] Webserver started at http://127.30.148.190:39843/ using document root <none> and password file <none>
I20260619 16:54:33.923795 7487 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:33.923877 7487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:33.925122 7487 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:33.925822 7502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:33.925994 7487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:33.926093 7487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "ee48569a155a41d0b5483c3a1686e588"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "7b7d6c4908991d6d726168af414b6ef1"
server_key_iv: "c4ad3f69a35c1a2c6e8193b79e3d4ad0"
server_key_version: "encryptionkey@0"
I20260619 16:54:33.926409 7487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:33.945937 7487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:33.947995 7487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:33.948144 7487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:33.951843 7487 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:32943
I20260619 16:54:33.951910 7554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:32943 every 8 connection(s)
I20260619 16:54:33.952298 7487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:33.953708 7555 sys_catalog.cc:263] Verifying existing consensus state
I20260619 16:54:33.954311 7555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588: Bootstrap starting.
I20260619 16:54:33.957121 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 7487
I20260619 16:54:33.957510 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:41243
--local_ip_for_outbound_sockets=127.30.148.129
--tserver_master_addrs=127.30.148.190:32943
--webserver_port=41285
--webserver_interface=127.30.148.129
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.148.129
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:54:33.962720 7555 log.cc:824] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:33.964248 7555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:54:33.964519 7555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588: Bootstrap complete.
I20260619 16:54:33.966694 7555 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48569a155a41d0b5483c3a1686e588" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32943 } }
I20260619 16:54:33.966938 7555 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee48569a155a41d0b5483c3a1686e588, State: Initialized, Role: FOLLOWER
I20260619 16:54:33.967119 7555 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48569a155a41d0b5483c3a1686e588" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32943 } }
I20260619 16:54:33.967204 7555 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:33.967254 7555 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:33.967299 7555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 1 FOLLOWER]: Advancing to term 2
I20260619 16:54:33.968235 7555 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48569a155a41d0b5483c3a1686e588" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32943 } }
I20260619 16:54:33.968348 7555 leader_election.cc:302] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee48569a155a41d0b5483c3a1686e588; no voters:
I20260619 16:54:33.968550 7555 leader_election.cc:288] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 16:54:33.968577 7559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 16:54:33.968688 7559 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [term 2 LEADER]: Becoming Leader. State: Replica: ee48569a155a41d0b5483c3a1686e588, State: Running, Role: LEADER
I20260619 16:54:33.968792 7559 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48569a155a41d0b5483c3a1686e588" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32943 } }
I20260619 16:54:33.968799 7555 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:33.969050 7560 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ee48569a155a41d0b5483c3a1686e588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48569a155a41d0b5483c3a1686e588" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32943 } } }
I20260619 16:54:33.969139 7560 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:33.969338 7563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:33.969908 7563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:33.970222 7563 catalog_manager.cc:1259] Loaded cluster ID: f153ab8df0a34f5388385877e7a42375
I20260619 16:54:33.970494 7563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:33.971359 7563 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:33.971537 7563 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588: Loaded TSK: 0
I20260619 16:54:33.971741 7563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:54:33.969909 7561 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee48569a155a41d0b5483c3a1686e588. Latest consensus state: current_term: 2 leader_uuid: "ee48569a155a41d0b5483c3a1686e588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48569a155a41d0b5483c3a1686e588" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32943 } } }
I20260619 16:54:33.971864 7561 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee48569a155a41d0b5483c3a1686e588 [sys.catalog]: This master's current role is: LEADER
W20260619 16:54:34.055912 7557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:34.056089 7557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:34.056111 7557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:34.057250 7557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:34.057291 7557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:34.057317 7557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:34.058719 7557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.129.keytab
--principal=oryx/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:41243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=41285
--tserver_master_addrs=127.30.148.190:32943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:34.058943 7557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:34.059180 7557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:34.063916 7584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:34.063903 7582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:34.063903 7581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:34.064193 7557 server_base.cc:1034] running on GCE node
I20260619 16:54:34.064392 7557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:34.064627 7557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:34.065788 7557 hybrid_clock.cc:648] HybridClock initialized: now 1781888074065772 us; error 33 us; skew 500 ppm
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888074, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:34.068416 7557 init.cc:377] Logged in from keytab as oryx/127.30.148.129@KRBTEST.COM (short username oryx)
I20260619 16:54:34.068917 7557 webserver.cc:460] Webserver started at http://127.30.148.129:41285/ using document root <none> and password file <none>
I20260619 16:54:34.069228 7557 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:34.069315 7557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:34.071049 7557 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:34.071686 7591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:34.071888 7557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:34.071990 7557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "66bb6790154f43db97b53e91dc8b36fb"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "9689e050a7bab381f862a360a92f755e"
server_key_iv: "02204be1fd6ed849ef9e662e18ddd836"
server_key_version: "encryptionkey@0"
I20260619 16:54:34.072402 7557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:34.087863 7557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:34.090173 7557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:34.090312 7557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:34.090543 7557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:34.090863 7557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:34.090911 7557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:34.090945 7557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:34.090986 7557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:34.098272 7557 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:41243
I20260619 16:54:34.098304 7704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:41243 every 8 connection(s)
I20260619 16:54:34.098757 7557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:34.102592 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 7557
I20260619 16:54:34.102892 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:37837
--local_ip_for_outbound_sockets=127.30.148.130
--tserver_master_addrs=127.30.148.190:32943
--webserver_port=42007
--webserver_interface=127.30.148.130
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.148.130
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:34.110023 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:34.099621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:56079)
0619 16:54:34.099794 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:34.099835 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:34.100276 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:34.101892 (+ 1616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:34.101896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:34.102454 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:34.102637 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.102642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.103758 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.103759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.105090 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.105093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.105973 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.105974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.106117 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.106755 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:34.106766 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:34.109815 (+ 3049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
W20260619 16:54:34.110452 7705 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:32943 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.148.190:32943: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
W20260619 16:54:34.111023 7708 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:34.099702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:56079 (local address 127.30.148.190:32943)
0619 16:54:34.101489 (+ 1787us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:34.101497 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:34.101529 (+ 32us) server_negotiation.cc:407] Connection header received
0619 16:54:34.101650 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:34.101656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:34.101707 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:34.101852 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:34.102787 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.103650 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.105197 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.105865 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.105995 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.109976 (+ 3981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:56079: BlockingRecv error: recv got EOF from 127.30.148.129:56079 (error 108)
Metrics: {"server-negotiator.queue_time_us":1738,"thread_start_us":95,"threads_started":1}
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:34.116329 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:34.110697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:33125)
0619 16:54:34.110756 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:34.110768 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:34.110815 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:34.111070 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:34.111071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:34.111215 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:34.111264 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.111266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.112235 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.112236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.113141 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.113144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.113673 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.113674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.113742 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.114171 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:34.114177 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:34.116266 (+ 2089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 16:54:34.116437 7710 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:34.110796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:33125 (local address 127.30.148.190:32943)
0619 16:54:34.110930 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:34.110932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:34.110938 (+ 6us) server_negotiation.cc:407] Connection header received
0619 16:54:34.110961 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:34.110962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:34.110973 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:34.111337 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:34.111378 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.112140 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.113239 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.113579 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.113877 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.116369 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:33125: BlockingRecv error: recv got EOF from 127.30.148.129:33125 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:34.121737 7707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:34.116844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:46045)
0619 16:54:34.116895 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:34.116908 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:34.116950 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:34.117091 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:34.117092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:34.117223 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:34.117253 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.117254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.118144 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.118145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.118949 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.118951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.119399 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.119400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.119452 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.119875 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:34.119880 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:34.121674 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 16:54:34.121898 7705 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 16:54:34.121951 7710 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:34.116922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:46045 (local address 127.30.148.190:32943)
0619 16:54:34.116952 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:34.116953 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:34.116961 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:34.117008 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:34.117009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:34.117019 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:34.117080 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:34.117324 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.118062 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.119031 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.119326 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.119390 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.121904 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:46045: BlockingRecv error: recv got EOF from 127.30.148.129:46045 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
W20260619 16:54:34.186872 7709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:34.187091 7709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:34.187129 7709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:34.188328 7709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:34.188388 7709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:34.188433 7709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:34.189853 7709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.130.keytab
--principal=oryx/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:37837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=42007
--tserver_master_addrs=127.30.148.190:32943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:34.190106 7709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:34.190317 7709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:34.194952 7715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:34.194993 7716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:34.194939 7718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:34.195133 7709 server_base.cc:1034] running on GCE node
I20260619 16:54:34.195302 7709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:34.195583 7709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:34.196730 7709 hybrid_clock.cc:648] HybridClock initialized: now 1781888074196719 us; error 28 us; skew 500 ppm
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888074, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:34.199191 7709 init.cc:377] Logged in from keytab as oryx/127.30.148.130@KRBTEST.COM (short username oryx)
I20260619 16:54:34.199618 7709 webserver.cc:460] Webserver started at http://127.30.148.130:42007/ using document root <none> and password file <none>
I20260619 16:54:34.199787 7709 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:34.199836 7709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:34.201028 7709 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:34.201577 7725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:34.201746 7709 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:34.201845 7709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "38c89d42433d422596f8e52c903734a1"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "2ef13afd873d77d223f7908138084a64"
server_key_iv: "76814fae55a61f11a5fc3feb6515baba"
server_key_version: "encryptionkey@0"
I20260619 16:54:34.202158 7709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:34.218957 7709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:34.221112 7709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:34.221258 7709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:34.221508 7709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:34.221792 7709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:34.221841 7709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:34.221880 7709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:34.221907 7709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:34.227412 7709 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:37837
I20260619 16:54:34.227473 7838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:37837 every 8 connection(s)
I20260619 16:54:34.227764 7709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:54:34.228137 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 7709
I20260619 16:54:34.228479 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:42081
--local_ip_for_outbound_sockets=127.30.148.131
--tserver_master_addrs=127.30.148.190:32943
--webserver_port=42981
--webserver_interface=127.30.148.131
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.148.131
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:34.236291 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:34.228454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:48815)
0619 16:54:34.228648 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:34.228692 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:34.229416 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:34.229717 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:34.229721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:34.230505 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:34.230737 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.230743 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.231455 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.231457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.232791 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.232794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.233151 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.233152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.233311 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.233813 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:34.233823 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:34.236169 (+ 2346us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":107,"threads_started":1}
W20260619 16:54:34.236388 7710 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:34.228454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:48815 (local address 127.30.148.190:32943)
0619 16:54:34.228509 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:34.228512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:34.228768 (+ 256us) server_negotiation.cc:407] Connection header received
0619 16:54:34.229543 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:34.229545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:34.229561 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:34.229633 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:34.230840 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.231363 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.232869 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.233068 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.233122 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.236290 (+ 3168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:48815: BlockingRecv error: recv got EOF from 127.30.148.130:48815 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260619 16:54:34.236682 7839 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:32943 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.148.190:32943: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:34.240857 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:34.236940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:45409)
0619 16:54:34.236978 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:34.236987 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:34.237046 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:34.237190 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:34.237191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:34.237323 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:34.237394 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.237395 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.237966 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.237967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.238436 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.238438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.238806 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.238807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.238871 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.239256 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:34.239261 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:34.240786 (+ 1525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 16:54:34.240909 7710 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:34.236997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:45409 (local address 127.30.148.190:32943)
0619 16:54:34.237054 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:34.237056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:34.237081 (+ 25us) server_negotiation.cc:407] Connection header received
0619 16:54:34.237101 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:34.237101 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:34.237119 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:34.237171 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:34.237451 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.237878 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.238515 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.238725 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.238807 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.240850 (+ 2043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:45409: BlockingRecv error: recv got EOF from 127.30.148.130:45409 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:34.245218 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:34.241114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:36889)
0619 16:54:34.241146 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:34.241154 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:34.241185 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:34.241281 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:34.241282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:34.241442 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:34.241471 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.241472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.242080 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.242081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.242736 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.242739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.243119 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.243120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.243188 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.243577 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:34.243581 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:34.245146 (+ 1565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":21}
W20260619 16:54:34.245323 7839 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 16:54:34.245262 7710 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:34.241160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:36889 (local address 127.30.148.190:32943)
0619 16:54:34.241195 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:34.241197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:34.241205 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:34.241226 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:34.241227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:34.241235 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:34.241281 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:34.241564 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.241989 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.242850 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.243030 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.243117 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.245206 (+ 2089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:36889: BlockingRecv error: recv got EOF from 127.30.148.130:36889 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 16:54:34.309037 7842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:34.309217 7842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:34.309240 7842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:34.310387 7842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:34.310431 7842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:34.310456 7842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:34.311830 7842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/krb5kdc/oryx_127.30.148.131.keytab
--principal=oryx/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:42081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=42981
--tserver_master_addrs=127.30.148.190:32943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:34.312057 7842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:34.312256 7842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:34.317250 7847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:34.317261 7850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:34.317400 7842 server_base.cc:1034] running on GCE node
W20260619 16:54:34.317261 7848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:34.317744 7842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:34.317998 7842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:34.319136 7842 hybrid_clock.cc:648] HybridClock initialized: now 1781888074319114 us; error 38 us; skew 500 ppm
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888074, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:34.321622 7842 init.cc:377] Logged in from keytab as oryx/127.30.148.131@KRBTEST.COM (short username oryx)
I20260619 16:54:34.321997 7842 webserver.cc:460] Webserver started at http://127.30.148.131:42981/ using document root <none> and password file <none>
I20260619 16:54:34.322163 7842 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:34.322216 7842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:34.323401 7842 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:34.323906 7857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:34.324102 7842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:34.324198 7842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "ba5ac18b2fbf4004ac5861a41f0bd2f1"
format_stamp: "Formatted at 2026-06-19 16:54:33 on dist-test-slave-xb9n"
server_key: "8c5f13f2d7f6cace21403855f2d1db76"
server_key_iv: "d5a2955f69667195edf9480fd455a1f8"
server_key_version: "encryptionkey@0"
I20260619 16:54:34.324502 7842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:34.334993 7842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:34.337162 7842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:34.337317 7842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:34.337596 7842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:34.337911 7842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:34.337946 7842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:34.337966 7842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:34.338012 7842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:34.343556 7842 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:42081
I20260619 16:54:34.343631 7970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:42081 every 8 connection(s)
I20260619 16:54:34.343982 7842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:34.350967 7973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:34.344666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:41161)
0619 16:54:34.344833 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:34.344866 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:34.345347 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:34.345631 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:34.345635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:34.346179 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:34.346353 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.346358 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.347051 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.347052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.347838 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.347839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.348244 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.348245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.348367 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.348769 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:34.348776 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:34.350838 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
W20260619 16:54:34.350966 7710 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:34.344680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:41161 (local address 127.30.148.190:32943)
0619 16:54:34.344743 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:34.344745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:34.344953 (+ 208us) server_negotiation.cc:407] Connection header received
0619 16:54:34.345474 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:34.345476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:34.345487 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:34.345561 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:34.346462 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.346954 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.347941 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.348147 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.348234 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.350923 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:41161: BlockingRecv error: recv got EOF from 127.30.148.131:41161 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260619 16:54:34.351183 7971 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:32943 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.148.190:32943: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
I20260619 16:54:34.353809 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 7842
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:34.355475 7973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:34.351395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:52953)
0619 16:54:34.351444 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:34.351456 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:34.351499 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:34.351734 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:34.351735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:34.351838 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:34.351877 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.351878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.352428 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.352429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.352904 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.352905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.353191 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.353191 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.353238 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.353647 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:34.353653 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:34.355400 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 16:54:34.355537 7710 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:34.351579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:52953 (local address 127.30.148.190:32943)
0619 16:54:34.351629 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:34.351631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:34.351637 (+ 6us) server_negotiation.cc:407] Connection header received
0619 16:54:34.351653 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:34.351653 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:34.351661 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:34.351701 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:34.351939 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.352368 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.352967 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.353133 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.353179 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.355463 (+ 2284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:52953: BlockingRecv error: recv got EOF from 127.30.148.131:52953 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:34 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:34.360515 7973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:34.355952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:49239)
0619 16:54:34.356063 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:34.356074 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:34.356124 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:34.356234 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:34.356234 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:34.356363 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:34.356389 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.356390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.357077 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.357078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.357869 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:34.357872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.358186 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.358187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:34.358241 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.358598 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:34.358602 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:34.360444 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98}
W20260619 16:54:34.360589 7710 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:34.356006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:49239 (local address 127.30.148.190:32943)
0619 16:54:34.356054 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:34.356056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:34.356111 (+ 55us) server_negotiation.cc:407] Connection header received
0619 16:54:34.356176 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:34.356177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:34.356184 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:34.356222 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:34.356445 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.357008 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.357950 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.358114 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.358166 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.360504 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:49239: BlockingRecv error: recv got EOF from 127.30.148.131:49239 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 16:54:34.360757 7971 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 16:54:34.361572 7708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:34.354197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50256 (local address 127.30.148.190:32943)
0619 16:54:34.354250 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:34.354252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:34.354430 (+ 178us) server_negotiation.cc:407] Connection header received
0619 16:54:34.354472 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:34.354473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:34.354488 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:34.354544 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:34.355147 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.355681 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.356699 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:34.356976 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:34.357048 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:34.358399 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:34.358403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:34.358403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:34.358414 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:34.360129 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:34.360500 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:34.360503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:34.360504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:34.360665 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:34.360866 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:34.360868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:34.360868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:34.361093 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:34.361240 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:34.361375 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 16:54:34.361485 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Jun 19 16:54:35 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:35 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:35.127219 7975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:35.122262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:60191)
0619 16:54:35.122399 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:35.122410 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:35.122483 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:35.122629 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:35.122630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:35.122851 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:35.122921 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:35.122923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:35.123754 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:35.123756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:35.124296 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:35.124298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:35.124689 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:35.124691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:35.124754 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:35.125263 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:35.125268 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:35.127152 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
W20260619 16:54:35.127292 7976 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:35.122353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:60191 (local address 127.30.148.190:32943)
0619 16:54:35.122481 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:35.122483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:35.122491 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:35.122546 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:35.122547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:35.122556 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:35.122614 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:35.123016 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:35.123671 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:35.124384 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:35.124599 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:35.124687 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:35.127239 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:60191: BlockingRecv error: recv got EOF from 127.30.148.129:60191 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
Jun 19 16:54:35 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:35 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:35.250119 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:35.245876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:51223)
0619 16:54:35.246019 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:35.246029 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:35.246080 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:35.246195 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:35.246196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:35.246353 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:35.246398 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:35.246400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:35.247016 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:35.247016 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:35.247547 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:35.247548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:35.247853 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:35.247854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:35.247892 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:35.248365 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:35.248370 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:35.250058 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
W20260619 16:54:35.250257 7976 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:35.245984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:51223 (local address 127.30.148.190:32943)
0619 16:54:35.246061 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:35.246063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:35.246079 (+ 16us) server_negotiation.cc:407] Connection header received
0619 16:54:35.246131 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:35.246132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:35.246141 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:35.246181 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:35.246462 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:35.246954 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:35.247617 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:35.247795 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:35.247845 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:35.250161 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:51223: BlockingRecv error: recv got EOF from 127.30.148.130:51223 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 16:54:35 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:35 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:35.365808 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:35.361176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:38911)
0619 16:54:35.361348 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:35.361372 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:35.361453 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:35.361599 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:35.361600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:35.361772 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:35.361815 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:35.361817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:35.362525 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:35.362526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:35.363058 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:35.363059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:35.363473 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:35.363474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:35.363539 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:35.364013 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:35.364018 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:35.365712 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":96,"threads_started":1}
W20260619 16:54:35.365836 7976 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:35.361261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:38911 (local address 127.30.148.190:32943)
0619 16:54:35.361318 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:35.361320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:35.361441 (+ 121us) server_negotiation.cc:407] Connection header received
0619 16:54:35.361507 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:35.361508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:35.361522 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:35.361592 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:35.361896 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:35.362436 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:35.363156 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:35.363362 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:35.363473 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:35.365776 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:38911: BlockingRecv error: recv got EOF from 127.30.148.131:38911 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 16:54:36 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:36 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:36.132261 7979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:36.127680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:60391)
0619 16:54:36.127816 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:36.127826 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:36.127874 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:36.128020 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:36.128021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:36.128170 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:36.128219 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:36.128221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:36.128885 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:36.128886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:36.129466 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:36.129469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:36.129882 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:36.129883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:36.129944 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:36.130443 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:36.130449 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:36.132157 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
W20260619 16:54:36.132316 7980 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:36.127741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:60391 (local address 127.30.148.190:32943)
0619 16:54:36.127885 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:36.127887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:36.127894 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:54:36.127916 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:36.127917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:36.127926 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:36.127981 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:36.128273 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:36.128815 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:36.129583 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:36.129794 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:36.129879 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:36.132244 (+ 2365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:60391: BlockingRecv error: recv got EOF from 127.30.148.129:60391 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":95,"threads_started":1}
Jun 19 16:54:36 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:36 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:36.255473 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:36.250710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:49989)
0619 16:54:36.250898 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:36.250911 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:36.251008 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:36.251169 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:36.251170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:36.251345 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:36.251390 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:36.251392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:36.252073 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:36.252075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:36.252592 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:36.252594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:36.252954 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:36.252955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:36.253035 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:36.253676 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:36.253682 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:36.255419 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":110,"threads_started":1}
W20260619 16:54:36.255580 7980 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:36.250830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:49989 (local address 127.30.148.190:32943)
0619 16:54:36.250902 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:36.250904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:36.250993 (+ 89us) server_negotiation.cc:407] Connection header received
0619 16:54:36.251072 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:36.251073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:36.251084 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:36.251163 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:36.251473 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:36.251971 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:36.252674 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:36.252880 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:36.252957 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:36.255512 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:49989: BlockingRecv error: recv got EOF from 127.30.148.130:49989 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 16:54:36 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:36 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:36.371382 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:36.366510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:53597)
0619 16:54:36.366682 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:36.366694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:36.366763 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:36.366914 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:36.366915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:36.367084 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:36.367128 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:36.367129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:36.367809 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:36.367811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:36.368323 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:36.368325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:36.368780 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:36.368781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:36.368845 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:36.369321 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:36.369326 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:36.371284 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
W20260619 16:54:36.371480 7980 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:36.366636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:53597 (local address 127.30.148.190:32943)
0619 16:54:36.366714 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:36.366716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:36.366741 (+ 25us) server_negotiation.cc:407] Connection header received
0619 16:54:36.366833 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:36.366834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:36.366849 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:36.366905 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:36.367204 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:36.367722 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:36.368407 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:36.368687 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:36.368778 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:36.371390 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:53597: BlockingRecv error: recv got EOF from 127.30.148.131:53597 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 16:54:37 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:37 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:37.137072 7983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:37.132689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:35731)
0619 16:54:37.132844 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:37.132854 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:37.132917 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:37.133047 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:37.133047 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:37.133199 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:37.133248 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:37.133250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:37.133906 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:37.133906 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:37.134422 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:37.134424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:37.134731 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:37.134731 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:37.134769 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:37.135235 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:37.135241 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:37.137014 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
W20260619 16:54:37.137173 7984 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:37.132764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:35731 (local address 127.30.148.190:32943)
0619 16:54:37.132866 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:37.132867 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:37.132918 (+ 51us) server_negotiation.cc:407] Connection header received
0619 16:54:37.132960 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:37.132960 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:37.132971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:37.133030 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:37.133307 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:37.133842 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:37.134488 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:37.134671 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:37.134720 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:37.137097 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:35731: BlockingRecv error: recv got EOF from 127.30.148.129:35731 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":47,"threads_started":1}
Jun 19 16:54:37 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:37 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:37.260332 7985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:37.256017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:57845)
0619 16:54:37.256156 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:37.256168 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:37.256241 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:37.256375 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:37.256376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:37.256535 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:37.256580 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:37.256582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:37.257212 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:37.257214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:37.257804 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:37.257806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:37.258147 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:37.258148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:37.258186 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:37.258622 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:37.258626 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:37.260268 (+ 1642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
W20260619 16:54:37.260448 7984 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:37.256093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:57845 (local address 127.30.148.190:32943)
0619 16:54:37.256141 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:37.256143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:37.256242 (+ 99us) server_negotiation.cc:407] Connection header received
0619 16:54:37.256302 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:37.256302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:37.256313 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:37.256356 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:37.256640 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:37.257136 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:37.257885 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:37.258083 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:37.258139 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:37.260355 (+ 2216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:57845: BlockingRecv error: recv got EOF from 127.30.148.130:57845 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Jun 19 16:54:37 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:37 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:37.376168 7986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:37.371924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:38209)
0619 16:54:37.372080 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:37.372091 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:37.372181 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:37.372299 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:37.372300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:37.372468 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:37.372515 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:37.372517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:37.373141 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:37.373142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:37.373665 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:37.373666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:37.374015 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:37.374016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:37.374068 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:37.374448 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:37.374452 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:37.376104 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":80,"threads_started":1}
W20260619 16:54:37.376274 7984 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:37.372002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:38209 (local address 127.30.148.190:32943)
0619 16:54:37.372075 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:37.372077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:37.372165 (+ 88us) server_negotiation.cc:407] Connection header received
0619 16:54:37.372233 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:37.372234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:37.372247 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:37.372295 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:37.372590 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:37.373062 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:37.373742 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:37.373934 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:37.374009 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:37.376185 (+ 2176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:38209: BlockingRecv error: recv got EOF from 127.30.148.131:38209 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 16:54:38 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:38 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:38.141791 7987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:38.137547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:35021)
0619 16:54:38.137689 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:38.137707 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:38.137770 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:38.137905 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:38.137905 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:38.138065 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:38.138114 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:38.138115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:38.138740 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:38.138741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:38.139259 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:38.139261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:38.139590 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:38.139591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:38.139629 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:38.140063 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:38.140067 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:38.141733 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
W20260619 16:54:38.141893 7988 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:38.137611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:35021 (local address 127.30.148.190:32943)
0619 16:54:38.137729 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:38.137731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:38.137760 (+ 29us) server_negotiation.cc:407] Connection header received
0619 16:54:38.137814 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:38.137815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:38.137825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:38.137885 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:38.138172 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:38.138665 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:38.139331 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:38.139523 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:38.139579 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:38.141819 (+ 2240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:35021: BlockingRecv error: recv got EOF from 127.30.148.129:35021 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":71,"threads_started":1}
Jun 19 16:54:38 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:38 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:38.268359 7989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:38.260969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:52075)
0619 16:54:38.261109 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:38.261124 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:38.261178 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:38.261297 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:38.261297 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:38.261465 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:38.261512 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:38.261513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:38.262137 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:38.262137 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:38.262638 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:38.262640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:38.262987 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:38.263006 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:38.263068 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:38.266555 (+ 3487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:38.266561 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:38.268268 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
W20260619 16:54:38.268389 7988 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:38.261051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:52075 (local address 127.30.148.190:32943)
0619 16:54:38.261099 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:38.261102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:38.261167 (+ 65us) server_negotiation.cc:407] Connection header received
0619 16:54:38.261237 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:38.261238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:38.261249 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:38.261286 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:38.261571 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:38.262070 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:38.262710 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:38.262912 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:38.262976 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:38.268333 (+ 5357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:52075: BlockingRecv error: recv got EOF from 127.30.148.130:52075 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 16:54:38 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:38 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:38.380846 7990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:38.376587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:49723)
0619 16:54:38.376726 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:38.376741 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:38.376794 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:38.376926 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:38.376927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:38.377079 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:38.377126 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:38.377128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:38.377848 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:38.377849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:38.378352 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:38.378354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:38.378666 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:38.378666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:38.378703 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:38.379132 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:38.379136 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:38.380791 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
W20260619 16:54:38.380988 7988 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:38.376648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:49723 (local address 127.30.148.190:32943)
0619 16:54:38.376718 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:38.376720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:38.376824 (+ 104us) server_negotiation.cc:407] Connection header received
0619 16:54:38.376846 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:38.376846 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:38.376857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:38.376907 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:38.377185 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:38.377783 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:38.378428 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:38.378602 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:38.378656 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:38.380907 (+ 2251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:49723: BlockingRecv error: recv got EOF from 127.30.148.131:49723 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 19 16:54:39 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:39 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:39.146716 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:39.142371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:51425)
0619 16:54:39.142533 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:39.142548 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:39.142595 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:39.142748 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:39.142749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:39.142898 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:39.142947 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:39.142949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:39.143564 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:39.143565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:39.144076 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:39.144078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:39.144396 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:39.144397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:39.144437 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:39.144868 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:39.144872 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:39.146649 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
W20260619 16:54:39.146838 7992 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:39.142428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:51425 (local address 127.30.148.190:32943)
0619 16:54:39.142570 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:39.142572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:39.142590 (+ 18us) server_negotiation.cc:407] Connection header received
0619 16:54:39.142655 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:39.142656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:39.142666 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:39.142726 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:39.143008 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:39.143500 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:39.144151 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:39.144332 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:39.144382 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:39.146737 (+ 2355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:51425: BlockingRecv error: recv got EOF from 127.30.148.129:51425 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":102,"threads_started":1}
Jun 19 16:54:39 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:39 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:39.273717 7993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:39.268840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:47277)
0619 16:54:39.269001 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:39.269013 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:39.269073 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:39.269250 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:39.269251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:39.269503 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:39.269561 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:39.269564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:39.270338 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:39.270339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:39.270923 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:39.270925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:39.271293 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:39.271294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:39.271347 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:39.271805 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:39.271809 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:39.273624 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
W20260619 16:54:39.273762 7992 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:39.269024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:47277 (local address 127.30.148.190:32943)
0619 16:54:39.269083 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:39.269085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:39.269095 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:39.269161 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:39.269161 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:39.269172 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:39.269245 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:39.269650 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:39.270225 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:39.271014 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:39.271208 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:39.271294 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:39.273701 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:47277: BlockingRecv error: recv got EOF from 127.30.148.130:47277 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 16:54:39 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:39 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:39.386240 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:39.381327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:54719)
0619 16:54:39.381521 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:39.381532 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:39.381620 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:39.381789 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:39.381790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:39.381980 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:39.382024 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:39.382026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:39.382682 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:39.382683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:39.383448 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:39.383450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:39.383826 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:39.383827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:39.383867 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:39.384268 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:39.384272 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:39.386138 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":128,"threads_started":1}
W20260619 16:54:39.386278 7992 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:39.381428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:54719 (local address 127.30.148.190:32943)
0619 16:54:39.381494 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:39.381496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:39.381600 (+ 104us) server_negotiation.cc:407] Connection header received
0619 16:54:39.381681 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:39.381682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:39.381704 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:39.381769 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:39.382102 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:39.382598 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:39.383529 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:39.383750 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:39.383818 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:39.386218 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:54719: BlockingRecv error: recv got EOF from 127.30.148.131:54719 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 19 16:54:40 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:40 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:40.151503 7995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:40.147155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:49795)
0619 16:54:40.147288 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:40.147298 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:40.147343 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:40.147569 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:40.147570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:40.147723 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:40.147771 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:40.147772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:40.148464 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:40.148465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:40.148985 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:40.148986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:40.149309 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:40.149309 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:40.149348 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:40.149812 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:40.149816 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:40.151446 (+ 1630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
W20260619 16:54:40.151572 7996 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:40.147215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:49795 (local address 127.30.148.190:32943)
0619 16:54:40.147434 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:40.147435 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:40.147442 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:54:40.147477 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:40.147478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:40.147487 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:40.147546 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:40.147888 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:40.148394 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:40.149061 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:40.149245 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:40.149295 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:40.151519 (+ 2224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:49795: BlockingRecv error: recv got EOF from 127.30.148.129:49795 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":165,"threads_started":1}
Jun 19 16:54:40 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:40 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:40.278692 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:40.274254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:44051)
0619 16:54:40.274401 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:40.274412 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:40.274484 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:40.274621 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:40.274622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:40.274784 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:40.274834 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:40.274835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:40.275463 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:40.275464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:40.276083 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:40.276085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:40.276400 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:40.276401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:40.276442 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:40.276867 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:40.276872 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:40.278635 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
W20260619 16:54:40.278788 7996 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:40.274323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:44051 (local address 127.30.148.190:32943)
0619 16:54:40.274376 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:40.274379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:40.274486 (+ 107us) server_negotiation.cc:407] Connection header received
0619 16:54:40.274539 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:40.274540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:40.274551 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:40.274602 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:40.274895 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:40.275387 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:40.276151 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:40.276335 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:40.276389 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:40.278721 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:44051: BlockingRecv error: recv got EOF from 127.30.148.130:44051 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Jun 19 16:54:40 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:40 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:40.391214 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:40.386690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:41443)
0619 16:54:40.386886 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:40.386898 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:40.386992 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:40.387167 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:40.387168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:40.387345 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:40.387393 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:40.387394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:40.388080 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:40.388081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:40.388645 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:40.388647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:40.389035 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:40.389036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:40.389098 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:40.389534 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:40.389538 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:40.391149 (+ 1611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":102,"threads_started":1}
W20260619 16:54:40.391328 7996 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:40.386792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:41443 (local address 127.30.148.190:32943)
0619 16:54:40.386871 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:40.386873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:40.386974 (+ 101us) server_negotiation.cc:407] Connection header received
0619 16:54:40.387071 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:40.387072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:40.387088 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:40.387158 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:40.387485 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:40.387994 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:40.388734 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:40.388922 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:40.389018 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:40.391234 (+ 2216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:41443: BlockingRecv error: recv got EOF from 127.30.148.131:41443 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 16:54:41 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:41 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:41.156739 7999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:41.152075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:45545)
0619 16:54:41.152200 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:41.152211 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:41.152271 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:41.152404 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:41.152405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:41.152558 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:41.152624 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:41.152626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:41.153450 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:41.153451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:41.154060 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:41.154062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:41.154400 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:41.154400 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:41.154436 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:41.154882 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:41.154886 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:41.156684 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
W20260619 16:54:41.156872 8000 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:41.152147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:45545 (local address 127.30.148.190:32943)
0619 16:54:41.152275 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:41.152276 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:41.152306 (+ 30us) server_negotiation.cc:407] Connection header received
0619 16:54:41.152326 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:41.152327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:41.152337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:41.152387 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:41.152685 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:41.153348 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:41.154124 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:41.154340 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:41.154393 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:41.156768 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:45545: BlockingRecv error: recv got EOF from 127.30.148.129:45545 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
Jun 19 16:54:41 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:41 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 16:54:41.283728 8000 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:41.279206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:53255 (local address 127.30.148.190:32943)
0619 16:54:41.279258 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:41.279260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:41.279339 (+ 79us) server_negotiation.cc:407] Connection header received
0619 16:54:41.279388 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:41.279389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:41.279400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:41.279449 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:41.279713 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:41.280191 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:41.280820 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:41.280996 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:41.281046 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:41.283678 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:53255: BlockingRecv error: recv got EOF from 127.30.148.130:53255 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 16:54:41.283774 8001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:41.279134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:53255)
0619 16:54:41.279274 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:41.279285 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:41.279345 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:41.279465 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:41.279466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:41.279616 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:41.279660 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:41.279662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:41.280253 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:41.280253 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:41.280758 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:41.280760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:41.281053 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:41.281054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:41.281091 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:41.281700 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:41.281715 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:41.283719 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
Jun 19 16:54:41 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:41 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:41.396016 8002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:41.391752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:46065)
0619 16:54:41.391898 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:41.391915 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:41.391992 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:41.392159 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:41.392160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:41.392361 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:41.392405 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:41.392406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:41.393065 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:41.393066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:41.393600 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:41.393602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:41.393939 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:41.393940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:41.393999 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:41.394410 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:41.394414 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:41.395963 (+ 1549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":53,"threads_started":1}
W20260619 16:54:41.396165 8000 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:41.391913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:46065 (local address 127.30.148.190:32943)
0619 16:54:41.391969 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:41.391971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:41.391979 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:41.392080 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:41.392080 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:41.392093 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:41.392153 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:41.392478 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:41.392988 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:41.393679 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:41.393862 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:41.393939 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:41.396039 (+ 2100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:46065: BlockingRecv error: recv got EOF from 127.30.148.131:46065 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 16:54:42 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:42 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:42.161441 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:42.157242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:43033)
0619 16:54:42.157427 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:42.157438 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:42.157487 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:42.157617 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:42.157618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:42.157772 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:42.157820 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:42.157822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:42.158435 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:42.158436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:42.158944 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:42.158945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:42.159250 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:42.159250 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:42.159289 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:42.159718 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:42.159722 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:42.161336 (+ 1614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":96,"threads_started":1}
W20260619 16:54:42.161538 8004 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:42.157393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:43033 (local address 127.30.148.190:32943)
0619 16:54:42.157505 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:42.157507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:42.157515 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:42.157536 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:42.157536 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:42.157545 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:42.157603 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:42.157883 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:42.158374 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:42.159009 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:42.159192 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:42.159241 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:42.161462 (+ 2221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:43033: BlockingRecv error: recv got EOF from 127.30.148.129:43033 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":58,"threads_started":1}
Jun 19 16:54:42 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:42 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:42.288887 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:42.284255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:55531)
0619 16:54:42.284389 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:42.284401 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:42.284456 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:42.284605 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:42.284606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:42.284786 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:42.284831 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:42.284833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:42.285701 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:42.285702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:42.286227 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:42.286229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:42.286600 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:42.286601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:42.286654 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:42.287129 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:42.287134 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:42.288796 (+ 1662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
W20260619 16:54:42.288925 8004 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:42.284411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:55531 (local address 127.30.148.190:32943)
0619 16:54:42.284480 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:42.284482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:42.284493 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:54:42.284519 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:42.284520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:42.284536 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:42.284597 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:42.284913 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:42.285617 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:42.286306 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:42.286515 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:42.286599 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:42.288867 (+ 2268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:55531: BlockingRecv error: recv got EOF from 127.30.148.130:55531 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 16:54:42 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:42 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:42.401002 8006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:42.396478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:44701)
0619 16:54:42.396628 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:42.396641 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:42.396734 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:42.396864 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:42.396865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:42.397051 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:42.397114 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:42.397115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:42.397841 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:42.397842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:42.398358 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:42.398360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:42.398710 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:42.398711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:42.398762 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:42.399263 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:42.399268 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:42.400946 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":76,"threads_started":1}
W20260619 16:54:42.401155 8004 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:42.396569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:44701 (local address 127.30.148.190:32943)
0619 16:54:42.396632 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:42.396635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:42.396695 (+ 60us) server_negotiation.cc:407] Connection header received
0619 16:54:42.396789 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:42.396790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:42.396804 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:42.396849 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:42.397194 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:42.397755 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:42.398450 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:42.398644 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:42.398713 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:42.401047 (+ 2334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:44701: BlockingRecv error: recv got EOF from 127.30.148.131:44701 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 16:54:43 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:43 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:43.166635 8007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:43.162055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:56695)
0619 16:54:43.162206 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:43.162220 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:43.162284 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:43.162420 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:43.162421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:43.162585 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:43.162634 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:43.162635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:43.163247 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:43.163248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:43.163872 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:43.163875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:43.164211 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:43.164212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:43.164250 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:43.164714 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:43.164718 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:43.166568 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
W20260619 16:54:43.166757 8008 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:43.162115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:56695 (local address 127.30.148.190:32943)
0619 16:54:43.162263 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:43.162265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:43.162273 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:43.162338 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:43.162338 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:43.162349 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:43.162406 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:43.162693 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:43.163173 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:43.163949 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:43.164152 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:43.164200 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:43.166654 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:56695: BlockingRecv error: recv got EOF from 127.30.148.129:56695 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
Jun 19 16:54:43 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:43 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:43.297046 8009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:43.289343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:53125)
0619 16:54:43.289524 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:43.289536 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:43.289624 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:43.289768 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:43.289770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:43.289942 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:43.289988 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:43.289989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:43.290653 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:43.290655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:43.291153 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:43.291155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:43.291500 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:43.291501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:43.291563 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:43.295263 (+ 3700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:43.295268 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:43.296946 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
W20260619 16:54:43.297083 8008 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:43.289455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:53125 (local address 127.30.148.190:32943)
0619 16:54:43.289521 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:43.289524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:43.289605 (+ 81us) server_negotiation.cc:407] Connection header received
0619 16:54:43.289681 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:43.289682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:43.289695 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:43.289761 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:43.290062 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:43.290575 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:43.291237 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:43.291422 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:43.291492 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:43.297023 (+ 5531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:53125: BlockingRecv error: recv got EOF from 127.30.148.130:53125 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 16:54:43 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:43 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:43.406204 8010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:43.401497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:55555)
0619 16:54:43.401627 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:43.401639 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:43.401715 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:43.401848 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:43.401848 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:43.402018 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:43.402063 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:43.402065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:43.402797 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:43.402798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:43.403313 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:43.403315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:43.403691 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:43.403692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:43.403761 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:43.404292 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:43.404296 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:43.406101 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
W20260619 16:54:43.406241 8008 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:43.401529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:55555 (local address 127.30.148.190:32943)
0619 16:54:43.401590 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:43.401593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:43.401703 (+ 110us) server_negotiation.cc:407] Connection header received
0619 16:54:43.401771 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:43.401772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:43.401785 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:43.401841 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:43.402139 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:43.402715 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:43.403399 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:43.403607 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:43.403690 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:43.406179 (+ 2489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:55555: BlockingRecv error: recv got EOF from 127.30.148.131:55555 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 16:54:44 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:44 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:44.171885 8011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:44.167060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:44965)
0619 16:54:44.167309 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:44.167319 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:44.167377 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:44.167512 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:44.167513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:44.167669 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:44.167729 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:44.167731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:44.168360 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:44.168360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:44.168962 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:44.168965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:44.169332 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:44.169333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:44.169417 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:44.169918 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:44.169923 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:44.171792 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":187,"threads_started":1}
W20260619 16:54:44.171929 8012 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:44.167224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:44965 (local address 127.30.148.190:32943)
0619 16:54:44.167339 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:44.167341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:44.167363 (+ 22us) server_negotiation.cc:407] Connection header received
0619 16:54:44.167420 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:44.167421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:44.167430 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:44.167499 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:44.167790 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:44.168276 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:44.169027 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:44.169254 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:44.169312 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:44.171861 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:44965: BlockingRecv error: recv got EOF from 127.30.148.129:44965 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":45,"threads_started":1}
Jun 19 16:54:44 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:44 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:44.302351 8013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:44.297618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:37287)
0619 16:54:44.297785 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:44.297798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:44.297885 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:44.298054 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:44.298054 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:44.298224 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:44.298268 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:44.298269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:44.298917 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:44.298918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:44.299743 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:44.299746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:44.300103 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:44.300104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:44.300156 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:44.300579 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:44.300584 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:44.302258 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
W20260619 16:54:44.302390 8012 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:44.297724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:37287 (local address 127.30.148.190:32943)
0619 16:54:44.297798 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:44.297800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:44.297864 (+ 64us) server_negotiation.cc:407] Connection header received
0619 16:54:44.297954 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:44.297955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:44.297978 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:44.298047 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:44.298348 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:44.298843 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:44.299824 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:44.300026 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:44.300095 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:44.302331 (+ 2236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:37287: BlockingRecv error: recv got EOF from 127.30.148.130:37287 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 16:54:44 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:44 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:44.410959 8014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:44.406661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:37939)
0619 16:54:44.406822 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:44.406833 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:44.406887 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:44.406995 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:44.406995 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:44.407139 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:44.407187 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:44.407189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:44.407807 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:44.407808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:44.408322 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:44.408324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:44.408629 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:44.408630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:44.408668 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:44.409179 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:44.409184 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:44.410902 (+ 1718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
W20260619 16:54:44.411082 8012 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:44.406776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:37939 (local address 127.30.148.190:32943)
0619 16:54:44.406823 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:44.406825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:44.406873 (+ 48us) server_negotiation.cc:407] Connection header received
0619 16:54:44.406927 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:44.406927 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:44.406938 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:44.406984 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:44.407242 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:44.407743 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:44.408385 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:44.408568 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:44.408617 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:44.410997 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:37939: BlockingRecv error: recv got EOF from 127.30.148.131:37939 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Jun 19 16:54:45 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:45 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:45.176470 8015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:45.172306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:33651)
0619 16:54:45.172415 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:45.172427 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:45.172494 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:45.172609 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:45.172610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:45.172768 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:45.172815 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:45.172817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:45.173434 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:45.173435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:45.173955 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:45.173956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:45.174259 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:45.174260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:45.174298 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:45.174742 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:45.174746 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:45.176405 (+ 1659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":51,"threads_started":1}
W20260619 16:54:45.176596 8016 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:45.172377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:33651 (local address 127.30.148.190:32943)
0619 16:54:45.172502 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:45.172503 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:45.172513 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:54:45.172536 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:45.172537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:45.172546 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:45.172604 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:45.172874 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:45.173381 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:45.174017 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:45.174201 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:45.174250 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:45.176516 (+ 2266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:33651: BlockingRecv error: recv got EOF from 127.30.148.129:33651 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
Jun 19 16:54:45 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:45 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:45.307415 8017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:45.302892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:51559)
0619 16:54:45.303031 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:45.303042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:45.303121 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:45.303254 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:45.303255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:45.303409 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:45.303454 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:45.303455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:45.304052 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:45.304053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:45.304618 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:45.304620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:45.304979 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:45.304980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:45.305033 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:45.305506 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:45.305510 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:45.307348 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
W20260619 16:54:45.307488 8016 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:45.302949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:51559 (local address 127.30.148.190:32943)
0619 16:54:45.303010 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:45.303012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:45.303118 (+ 106us) server_negotiation.cc:407] Connection header received
0619 16:54:45.303179 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:45.303179 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:45.303189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:45.303241 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:45.303508 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:45.303994 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:45.304680 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:45.304908 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:45.304961 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:45.307433 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:51559: BlockingRecv error: recv got EOF from 127.30.148.130:51559 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 16:54:45 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:45 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:45.415732 8018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:45.411440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:34001)
0619 16:54:45.411573 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:45.411581 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:45.411629 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:45.411744 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:45.411745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:45.411888 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:45.411933 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:45.411935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:45.412507 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:45.412508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:45.413025 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:45.413027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:45.413310 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:45.413311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:45.413347 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:45.413848 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:45.413852 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:45.415663 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
W20260619 16:54:45.415812 8016 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:45.411494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:34001 (local address 127.30.148.190:32943)
0619 16:54:45.411551 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:45.411553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:45.411627 (+ 74us) server_negotiation.cc:407] Connection header received
0619 16:54:45.411669 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:45.411670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:45.411681 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:45.411732 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:45.411981 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:45.412446 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:45.413085 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:45.413253 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:45.413300 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:45.415751 (+ 2451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:34001: BlockingRecv error: recv got EOF from 127.30.148.131:34001 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 16:54:46 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:46 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:46.181147 8019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:46.176885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:38269)
0619 16:54:46.176998 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:46.177010 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:46.177090 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:46.177244 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:46.177245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:46.177430 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:46.177484 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:46.177485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:46.178122 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:46.178123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:46.178664 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:46.178666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:46.178977 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:46.178977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:46.179017 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:46.179441 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:46.179445 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:46.181092 (+ 1647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":56,"threads_started":1}
W20260619 16:54:46.181253 8020 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:46.176943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:38269 (local address 127.30.148.190:32943)
0619 16:54:46.177084 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:46.177086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:46.177094 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:46.177164 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:46.177164 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:46.177174 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:46.177229 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:46.177554 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:46.178060 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:46.178731 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:46.178917 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:46.178968 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:46.181176 (+ 2208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:38269: BlockingRecv error: recv got EOF from 127.30.148.129:38269 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
Jun 19 16:54:46 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:46 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:46.312184 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:46.307851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:43367)
0619 16:54:46.307979 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:46.307988 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:46.308042 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:46.308170 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:46.308171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:46.308342 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:46.308388 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:46.308389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:46.309014 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:46.309015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:46.309618 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:46.309621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:46.309920 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:46.309921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:46.309974 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:46.310399 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:46.310403 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:46.312128 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
W20260619 16:54:46.312314 8020 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:46.307937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:43367 (local address 127.30.148.190:32943)
0619 16:54:46.308001 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:46.308003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:46.308030 (+ 27us) server_negotiation.cc:407] Connection header received
0619 16:54:46.308101 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:46.308102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:46.308111 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:46.308154 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:46.308453 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:46.308954 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:46.309683 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:46.309861 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:46.309912 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:46.312231 (+ 2319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:43367: BlockingRecv error: recv got EOF from 127.30.148.130:43367 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 16:54:46 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:46 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:46.420733 8022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:46.416283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:51525)
0619 16:54:46.416409 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:46.416421 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:46.416481 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:46.416630 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:46.416631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:46.416809 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:46.416852 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:46.416854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:46.417536 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:46.417537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:46.418045 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:46.418046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:46.418388 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:46.418388 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:46.418467 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:46.418901 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:46.418905 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:46.420632 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
W20260619 16:54:46.420758 8020 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:46.416387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:51525 (local address 127.30.148.190:32943)
0619 16:54:46.416447 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:46.416450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:46.416463 (+ 13us) server_negotiation.cc:407] Connection header received
0619 16:54:46.416551 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:46.416552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:46.416569 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:46.416625 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:46.416934 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:46.417461 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:46.418134 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:46.418320 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:46.418390 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:46.420702 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:51525: BlockingRecv error: recv got EOF from 127.30.148.131:51525 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 16:54:47 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:47 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:47.185976 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:47.181577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:42589)
0619 16:54:47.181714 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:47.181725 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:47.181780 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:47.181901 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:47.181901 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:47.182070 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:47.182134 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:47.182136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:47.182812 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:47.182813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:47.183342 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:47.183344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:47.183641 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:47.183642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:47.183679 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:47.184098 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:47.184102 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:47.185907 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
W20260619 16:54:47.186061 8024 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:47.181640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:42589 (local address 127.30.148.190:32943)
0619 16:54:47.181744 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:47.181746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:47.181766 (+ 20us) server_negotiation.cc:407] Connection header received
0619 16:54:47.181829 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:47.181830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:47.181840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:47.181888 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:47.182204 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:47.182732 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:47.183409 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:47.183585 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:47.183633 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:47.185991 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:42589: BlockingRecv error: recv got EOF from 127.30.148.129:42589 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":60,"threads_started":1}
Jun 19 16:54:47 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:47 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:47.316874 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:47.312634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:34687)
0619 16:54:47.312767 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:47.312783 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:47.312837 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:47.312958 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:47.312959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:47.313117 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:47.313162 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:47.313163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:47.313814 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:47.313815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:47.314344 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:47.314345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:47.314653 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:47.314654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:47.314693 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:47.315152 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:47.315157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:47.316788 (+ 1631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
W20260619 16:54:47.316972 8024 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:47.312733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:34687 (local address 127.30.148.190:32943)
0619 16:54:47.312775 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:47.312777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:47.312825 (+ 48us) server_negotiation.cc:407] Connection header received
0619 16:54:47.312882 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:47.312883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:47.312892 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:47.312942 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:47.313223 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:47.313727 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:47.314416 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:47.314594 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:47.314644 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:47.316889 (+ 2245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:34687: BlockingRecv error: recv got EOF from 127.30.148.130:34687 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 16:54:47 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:47 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:47.425278 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:47.421161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:55997)
0619 16:54:47.421279 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:47.421291 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:47.421374 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:47.421506 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:47.421507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:47.421668 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:47.421715 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:47.421717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:47.422336 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:47.422336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:47.422864 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:47.422865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:47.423161 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:47.423162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:47.423200 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:47.423616 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:47.423620 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:47.425215 (+ 1595us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":53,"threads_started":1}
W20260619 16:54:47.425391 8024 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:47.421229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:55997 (local address 127.30.148.190:32943)
0619 16:54:47.421302 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:47.421304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:47.421344 (+ 40us) server_negotiation.cc:407] Connection header received
0619 16:54:47.421440 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:47.421441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:47.421452 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:47.421493 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:47.421776 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:47.422274 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:47.422930 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:47.423105 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:47.423153 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:47.425307 (+ 2154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:55997: BlockingRecv error: recv got EOF from 127.30.148.131:55997 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 19 16:54:48 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:48 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:48.190960 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:48.186451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:51215)
0619 16:54:48.186609 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:48.186619 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:48.186686 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:48.186823 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:48.186824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:48.186984 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:48.187032 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:48.187033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:48.187682 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:48.187682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:48.188194 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:48.188196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:48.188500 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:48.188501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:48.188548 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:48.188974 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:48.188979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:48.190904 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
W20260619 16:54:48.191071 8028 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:48.186538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:51215 (local address 127.30.148.190:32943)
0619 16:54:48.186641 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:48.186642 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:48.186671 (+ 29us) server_negotiation.cc:407] Connection header received
0619 16:54:48.186731 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:48.186731 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:48.186746 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:48.186803 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:48.187096 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:48.187620 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:48.188258 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:48.188442 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:48.188502 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:48.190994 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:51215: BlockingRecv error: recv got EOF from 127.30.148.129:51215 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":46,"threads_started":1}
Jun 19 16:54:48 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:48 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:48.324796 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:48.317430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:55237)
0619 16:54:48.317597 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:48.317607 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:48.317681 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:48.317815 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:48.317816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:48.317993 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:48.318037 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:48.318039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:48.318689 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:48.318691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:48.319205 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:48.319207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:48.319573 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:48.319574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:48.319633 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:48.323026 (+ 3393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:48.323030 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:48.324738 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":64,"threads_started":1}
W20260619 16:54:48.324889 8028 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:48.317461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:55237 (local address 127.30.148.190:32943)
0619 16:54:48.317508 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:48.317510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:48.317670 (+ 160us) server_negotiation.cc:407] Connection header received
0619 16:54:48.317752 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:48.317753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:48.317765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:48.317812 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:48.318110 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:48.318610 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:48.319284 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:48.319492 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:48.319572 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:48.324828 (+ 5256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:55237: BlockingRecv error: recv got EOF from 127.30.148.130:55237 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 16:54:48 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:48 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.131@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:48.430100 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:48.425744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.131:41343)
0619 16:54:48.425890 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:48.425899 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:48.425953 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:48.426069 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:48.426069 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:48.426232 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:48.426278 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:48.426279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:48.426958 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:48.426959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:48.427498 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:48.427500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:48.427835 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:48.427835 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:48.427873 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:48.428295 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:48.428299 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:48.430045 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
W20260619 16:54:48.430230 8028 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:48.425795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:41343 (local address 127.30.148.190:32943)
0619 16:54:48.425858 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:48.425860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:48.425950 (+ 90us) server_negotiation.cc:407] Connection header received
0619 16:54:48.425996 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:48.425996 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:48.426008 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:48.426063 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:48.426333 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:48.426880 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:48.427564 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:48.427768 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:48.427829 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:48.430148 (+ 2319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.131:41343: BlockingRecv error: recv got EOF from 127.30.148.131:41343 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 16:54:49 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:49 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.129@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:49.195825 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:49.191546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.129:46263)
0619 16:54:49.191672 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:49.191684 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:49.191763 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:49.191892 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:49.191893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:49.192046 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:49.192094 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:49.192096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.192728 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.192728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:49.193240 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:49.193241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.193581 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.193582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:49.193624 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:49.194034 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:49.194039 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:49.195758 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":53,"threads_started":1}
W20260619 16:54:49.195919 8032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:49.191635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:46263 (local address 127.30.148.190:32943)
0619 16:54:49.191777 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:49.191779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:49.191788 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:54:49.191808 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:49.191809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:49.191819 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:49.191874 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:49.192154 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.192665 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.193307 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.193521 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.193572 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:49.195845 (+ 2273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.129:46263: BlockingRecv error: recv got EOF from 127.30.148.129:46263 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
Jun 19 16:54:49 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:54:49 dist-test-slave-xb9n krb5kdc[6974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.148.130@KRBTEST.COM for oryx/127.30.148.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:54:49.329810 8033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:49.325319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32943 (local address 127.30.148.130:60173)
0619 16:54:49.325470 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:49.325481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:49.325541 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:49.325658 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:49.325659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:49.325819 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:49.325865 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:49.325867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.326525 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.326526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:49.327054 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:49.327056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.327368 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.327369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:49.327435 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:49.327913 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:49.327918 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:49.329743 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.148.190:32943: Server oryx/127.30.148.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
W20260619 16:54:49.329919 8032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:49.325409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:60173 (local address 127.30.148.190:32943)
0619 16:54:49.325477 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:49.325479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:49.325534 (+ 55us) server_negotiation.cc:407] Connection header received
0619 16:54:49.325586 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:49.325587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:49.325596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:49.325643 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:49.325921 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.326455 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.327118 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.327307 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.327361 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:49.329831 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.148.130:60173: BlockingRecv error: recv got EOF from 127.30.148.130:60173 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 16:54:49.363984 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 7557
I20260619 16:54:49.368309 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 7709
I20260619 16:54:49.372404 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 7842
I20260619 16:54:49.376477 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 7487
2026-06-19T16:54:49Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17410 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T16:54:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:49Z Disabled control of system clock
I20260619 16:54:49.402848 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36017
--webserver_interface=127.30.148.190
--webserver_port=0
--builtin_ntp_servers=127.30.148.148:38019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:36017
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 16:54:49.480751 8037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:49.480937 8037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:49.480957 8037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:49.482115 8037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:49.482156 8037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:49.482168 8037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:49.482180 8037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:49.482193 8037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:49.483402 8037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:36017
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:49.483570 8037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:49.483784 8037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:49.488350 8042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:49.488500 8037 server_base.cc:1034] running on GCE node
W20260619 16:54:49.488380 8045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:49.488358 8043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:49.488838 8037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:49.489154 8037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:49.490303 8037 hybrid_clock.cc:648] HybridClock initialized: now 1781888089490297 us; error 34 us; skew 500 ppm
I20260619 16:54:49.491475 8037 webserver.cc:460] Webserver started at http://127.30.148.190:35039/ using document root <none> and password file <none>
I20260619 16:54:49.491695 8037 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:49.491760 8037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:49.491915 8037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:49.492872 8037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "fdad792113c7404aa961fe8a6dde1624"
format_stamp: "Formatted at 2026-06-19 16:54:49 on dist-test-slave-xb9n"
server_key: "8ccbed15c6bc586db03378d057e697e8"
server_key_iv: "2744a5656924949ff03b837c5f22f759"
server_key_version: "encryptionkey@0"
I20260619 16:54:49.493211 8037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "fdad792113c7404aa961fe8a6dde1624"
format_stamp: "Formatted at 2026-06-19 16:54:49 on dist-test-slave-xb9n"
server_key: "8ccbed15c6bc586db03378d057e697e8"
server_key_iv: "2744a5656924949ff03b837c5f22f759"
server_key_version: "encryptionkey@0"
I20260619 16:54:49.494450 8037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:49.495148 8051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:49.495299 8037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:49.495411 8037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "fdad792113c7404aa961fe8a6dde1624"
format_stamp: "Formatted at 2026-06-19 16:54:49 on dist-test-slave-xb9n"
server_key: "8ccbed15c6bc586db03378d057e697e8"
server_key_iv: "2744a5656924949ff03b837c5f22f759"
server_key_version: "encryptionkey@0"
I20260619 16:54:49.495486 8037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:49.507678 8037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:49.507923 8037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:49.508066 8037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:49.511873 8037 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:36017
I20260619 16:54:49.511901 8103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:36017 every 8 connection(s)
I20260619 16:54:49.512217 8037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:49.512809 8104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:49.515226 8104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624: Bootstrap starting.
I20260619 16:54:49.515729 8104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:49.515950 8104 log.cc:824] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:49.516479 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 8037
I20260619 16:54:49.516592 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:49.516655 8104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624: No bootstrap required, opened a new log
I20260619 16:54:49.516805 31314 external_mini_cluster.cc:1428] Setting key a6e1c73fec9672479a1952fa7dccbdc2
I20260619 16:54:49.518821 8104 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdad792113c7404aa961fe8a6dde1624" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36017 } }
I20260619 16:54:49.518944 8104 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:49.518990 8104 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdad792113c7404aa961fe8a6dde1624, State: Initialized, Role: FOLLOWER
I20260619 16:54:49.519116 8104 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdad792113c7404aa961fe8a6dde1624" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36017 } }
I20260619 16:54:49.519196 8104 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:49.519218 8104 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:49.519290 8104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:49.519793 8104 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdad792113c7404aa961fe8a6dde1624" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36017 } }
I20260619 16:54:49.519894 8104 leader_election.cc:302] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdad792113c7404aa961fe8a6dde1624; no voters:
I20260619 16:54:49.520103 8104 leader_election.cc:288] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:49.520128 8109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:49.520239 8109 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [term 1 LEADER]: Becoming Leader. State: Replica: fdad792113c7404aa961fe8a6dde1624, State: Running, Role: LEADER
I20260619 16:54:49.520324 8109 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdad792113c7404aa961fe8a6dde1624" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36017 } }
I20260619 16:54:49.520352 8104 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:49.520823 8111 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdad792113c7404aa961fe8a6dde1624. Latest consensus state: current_term: 1 leader_uuid: "fdad792113c7404aa961fe8a6dde1624" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdad792113c7404aa961fe8a6dde1624" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36017 } } }
I20260619 16:54:49.520913 8111 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:49.521147 8117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:49.521674 8117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:49.521694 8110 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdad792113c7404aa961fe8a6dde1624" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdad792113c7404aa961fe8a6dde1624" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36017 } } }
I20260619 16:54:49.522060 8110 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624 [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:49.523252 8117 catalog_manager.cc:1347] Generated new cluster ID: 3e383a3987134983890e162d7cb7c592
I20260619 16:54:49.523300 8117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:49.523573 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:49.517284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57774 (local address 127.30.148.190:36017)
0619 16:54:49.517517 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:49.517523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:49.517546 (+ 23us) server_negotiation.cc:407] Connection header received
0619 16:54:49.518004 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:49.518016 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:49.518070 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:49.518172 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:49.518431 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.519880 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.520938 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.521585 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.521703 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:49.522803 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:49.522806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:49.522807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:49.522990 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:49.523031 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:49.523045 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:49.523113 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:49.523345 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 16:54:49.523419 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":109,"threads_started":1}
I20260619 16:54:49.553347 8117 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:49.553869 8117 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:49.560618 8117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdad792113c7404aa961fe8a6dde1624: Generated new TSK 0
I20260619 16:54:49.560784 8117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:54:49.566560 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36017
--builtin_ntp_servers=127.30.148.148:38019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 16:54:49.669013 8128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:49.669200 8128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:49.669220 8128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:49.670320 8128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:49.670357 8128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:49.670382 8128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:49.671634 8128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:49.671840 8128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:49.672106 8128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:49.676753 8134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:49.676846 8128 server_base.cc:1034] running on GCE node
W20260619 16:54:49.676895 8136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:49.676777 8133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:49.677186 8128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:49.677449 8128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:49.678613 8128 hybrid_clock.cc:648] HybridClock initialized: now 1781888089678607 us; error 29 us; skew 500 ppm
I20260619 16:54:49.679665 8128 webserver.cc:460] Webserver started at http://127.30.148.129:33489/ using document root <none> and password file <none>
I20260619 16:54:49.679890 8128 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:49.679934 8128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:49.680068 8128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:49.681027 8128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "58d896cd40d04cab913437bd3f492194"
format_stamp: "Formatted at 2026-06-19 16:54:49 on dist-test-slave-xb9n"
server_key: "a4d542c3c78e553fe065f72f46f88667"
server_key_iv: "d547d765a0d4517a88126e732b235c74"
server_key_version: "encryptionkey@0"
I20260619 16:54:49.681402 8128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "58d896cd40d04cab913437bd3f492194"
format_stamp: "Formatted at 2026-06-19 16:54:49 on dist-test-slave-xb9n"
server_key: "a4d542c3c78e553fe065f72f46f88667"
server_key_iv: "d547d765a0d4517a88126e732b235c74"
server_key_version: "encryptionkey@0"
I20260619 16:54:49.682626 8128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:54:49.683218 8142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:49.683355 8128 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:49.683436 8128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "58d896cd40d04cab913437bd3f492194"
format_stamp: "Formatted at 2026-06-19 16:54:49 on dist-test-slave-xb9n"
server_key: "a4d542c3c78e553fe065f72f46f88667"
server_key_iv: "d547d765a0d4517a88126e732b235c74"
server_key_version: "encryptionkey@0"
I20260619 16:54:49.683512 8128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:49.698796 8128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:49.699043 8128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:49.699162 8128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:49.699343 8128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:49.699608 8128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:49.699653 8128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:49.699687 8128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:49.699712 8128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:49.705267 8128 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:41615
I20260619 16:54:49.705300 8255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:41615 every 8 connection(s)
I20260619 16:54:49.705603 8128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:49.710357 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:49.706214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:51291 (local address 127.30.148.190:36017)
0619 16:54:49.706254 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:49.706255 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:49.706498 (+ 243us) server_negotiation.cc:407] Connection header received
0619 16:54:49.706957 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:49.706958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:49.706972 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:49.707024 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:49.707657 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.708142 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.709088 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.709273 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.709342 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:49.710087 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:49.710088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:49.710089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:49.710094 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:49.710107 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:49.710111 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:49.710175 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:49.710256 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 16:54:49.710281 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:54:49.710585 8258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:49.706211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36017 (local address 127.30.148.129:51291)
0619 16:54:49.706385 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:49.706415 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:49.706863 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:49.707092 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:49.707096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:49.707105 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:49.707569 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:49.707573 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.708232 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.708234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:49.708998 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:49.709000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.709351 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.709376 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:49.709496 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:49.709957 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:49.709957 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:49.709988 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:49.710008 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:49.710174 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:49.710180 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:49.710183 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:49.710259 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 16:54:49.710354 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260619 16:54:49.711048 8256 heartbeater.cc:347] Connected to a master server at 127.30.148.190:36017
I20260619 16:54:49.711134 8256 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:49.711297 8256 heartbeater.cc:511] Master 127.30.148.190:36017 requested a full tablet report, sending...
I20260619 16:54:49.711723 8068 ts_manager.cc:194] Registered new tserver with Master: 58d896cd40d04cab913437bd3f492194 (127.30.148.129:41615)
I20260619 16:54:49.711925 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 8128
I20260619 16:54:49.712008 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:49.712188 31314 external_mini_cluster.cc:1428] Setting key 8eff68e9eda47f15ca4fdd056cd2ac4d
I20260619 16:54:49.712484 8068 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.148.129:51291
I20260619 16:54:49.713147 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36017
--builtin_ntp_servers=127.30.148.148:38019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 16:54:49.790197 8259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:49.790402 8259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:49.790436 8259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:49.791611 8259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:49.791671 8259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:49.791708 8259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:49.792990 8259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:49.793252 8259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:49.793568 8259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:49.798158 8264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:49.798188 8267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:49.798158 8265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:49.798560 8259 server_base.cc:1034] running on GCE node
I20260619 16:54:49.798738 8259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:49.798982 8259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:49.800133 8259 hybrid_clock.cc:648] HybridClock initialized: now 1781888089800128 us; error 50 us; skew 500 ppm
I20260619 16:54:49.801342 8259 webserver.cc:460] Webserver started at http://127.30.148.130:37875/ using document root <none> and password file <none>
I20260619 16:54:49.801647 8259 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:49.801692 8259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:49.801839 8259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:49.802812 8259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "dd127b04170242e89a38388161981fe9"
format_stamp: "Formatted at 2026-06-19 16:54:49 on dist-test-slave-xb9n"
server_key: "92bfd61dec6cf7c8db6893b0dfee47fe"
server_key_iv: "081a6653bd87888931e2f4f122beb5ac"
server_key_version: "encryptionkey@0"
I20260619 16:54:49.803134 8259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "dd127b04170242e89a38388161981fe9"
format_stamp: "Formatted at 2026-06-19 16:54:49 on dist-test-slave-xb9n"
server_key: "92bfd61dec6cf7c8db6893b0dfee47fe"
server_key_iv: "081a6653bd87888931e2f4f122beb5ac"
server_key_version: "encryptionkey@0"
I20260619 16:54:49.804315 8259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:54:49.804958 8273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:49.805132 8259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:49.805217 8259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "dd127b04170242e89a38388161981fe9"
format_stamp: "Formatted at 2026-06-19 16:54:49 on dist-test-slave-xb9n"
server_key: "92bfd61dec6cf7c8db6893b0dfee47fe"
server_key_iv: "081a6653bd87888931e2f4f122beb5ac"
server_key_version: "encryptionkey@0"
I20260619 16:54:49.805285 8259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:49.832504 8259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:49.832763 8259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:49.832903 8259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:49.833108 8259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:49.833467 8259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:49.833513 8259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:49.833544 8259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:49.833573 8259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:49.838958 8259 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:35461
I20260619 16:54:49.839000 8386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:35461 every 8 connection(s)
I20260619 16:54:49.839282 8259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:54:49.844904 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:49.839918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36017 (local address 127.30.148.130:33233)
0619 16:54:49.840100 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:49.840130 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:49.840601 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:49.840844 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:49.840848 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:49.840857 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:49.841312 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:49.841317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.842047 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.842048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:49.842938 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:49.842941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.843442 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.843443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:49.843587 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:49.844184 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:49.844185 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:49.844231 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:49.844256 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:49.844463 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:49.844469 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:49.844472 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:49.844576 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 16:54:49.844682 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260619 16:54:49.844998 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:49.839933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:33233 (local address 127.30.148.190:36017)
0619 16:54:49.839988 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:49.839990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:49.840213 (+ 223us) server_negotiation.cc:407] Connection header received
0619 16:54:49.840708 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:49.840709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:49.840722 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:49.840780 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:49.841446 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.841946 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.843038 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.843346 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.843425 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:49.844339 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:49.844341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:49.844341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:49.844346 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:49.844375 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:49.844380 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:49.844434 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:49.844873 (+ 439us) server_negotiation.cc:299] Negotiation successful
0619 16:54:49.844901 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 16:54:49.846149 8387 heartbeater.cc:347] Connected to a master server at 127.30.148.190:36017
I20260619 16:54:49.846251 8387 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:49.846413 8387 heartbeater.cc:511] Master 127.30.148.190:36017 requested a full tablet report, sending...
I20260619 16:54:49.846799 8067 ts_manager.cc:194] Registered new tserver with Master: dd127b04170242e89a38388161981fe9 (127.30.148.130:35461)
I20260619 16:54:49.847177 8067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.148.130:33233
I20260619 16:54:49.847929 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 8259
I20260619 16:54:49.847999 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:49.848201 31314 external_mini_cluster.cc:1428] Setting key b895fc37c646dde2f142b99af5c46dd4
I20260619 16:54:49.849054 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36017
--builtin_ntp_servers=127.30.148.148:38019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 16:54:49.925992 8390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:49.926199 8390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:49.926234 8390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:49.927292 8390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:49.927348 8390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:49.927388 8390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:49.928699 8390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:49.928936 8390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:49.929210 8390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:49.933836 8395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:49.933857 8396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:49.933857 8398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:49.934084 8390 server_base.cc:1034] running on GCE node
I20260619 16:54:49.934254 8390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:49.934412 8390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:49.935528 8390 hybrid_clock.cc:648] HybridClock initialized: now 1781888089935516 us; error 23 us; skew 500 ppm
I20260619 16:54:49.936651 8390 webserver.cc:460] Webserver started at http://127.30.148.131:40741/ using document root <none> and password file <none>
I20260619 16:54:49.936864 8390 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:49.936906 8390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:49.937039 8390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:49.938048 8390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "98160cf473594fd9be100caf661a8a6e"
format_stamp: "Formatted at 2026-06-19 16:54:49 on dist-test-slave-xb9n"
server_key: "2752ff7d86dab1f07be042467671f175"
server_key_iv: "7cf1c7cf929e0670cf0dad4fb2087001"
server_key_version: "encryptionkey@0"
I20260619 16:54:49.938380 8390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "98160cf473594fd9be100caf661a8a6e"
format_stamp: "Formatted at 2026-06-19 16:54:49 on dist-test-slave-xb9n"
server_key: "2752ff7d86dab1f07be042467671f175"
server_key_iv: "7cf1c7cf929e0670cf0dad4fb2087001"
server_key_version: "encryptionkey@0"
I20260619 16:54:49.939567 8390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:54:49.940251 8404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:49.940433 8390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:49.940513 8390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "98160cf473594fd9be100caf661a8a6e"
format_stamp: "Formatted at 2026-06-19 16:54:49 on dist-test-slave-xb9n"
server_key: "2752ff7d86dab1f07be042467671f175"
server_key_iv: "7cf1c7cf929e0670cf0dad4fb2087001"
server_key_version: "encryptionkey@0"
I20260619 16:54:49.940582 8390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:49.974547 8390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:49.974808 8390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:49.974949 8390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:49.975150 8390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:49.975462 8390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:49.975494 8390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:49.975543 8390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:49.975571 8390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:49.980834 8390 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:46875
I20260619 16:54:49.980880 8517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:46875 every 8 connection(s)
I20260619 16:54:49.981180 8390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:54:49.983742 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 8390
I20260619 16:54:49.983840 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:49.984045 31314 external_mini_cluster.cc:1428] Setting key 0d78d557acf09bda51ca686c5c5bdb5f
I20260619 16:54:49.986428 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:49.981855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:56657 (local address 127.30.148.190:36017)
0619 16:54:49.981916 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:49.981918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:49.982142 (+ 224us) server_negotiation.cc:407] Connection header received
0619 16:54:49.982585 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:49.982586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:49.982594 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:49.982650 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:49.983245 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.983697 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.985034 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.985279 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.985343 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:49.986163 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:49.986164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:49.986165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:49.986169 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:49.986183 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:49.986195 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:49.986242 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:49.986323 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 16:54:49.986355 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 16:54:49.986579 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:49.981831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36017 (local address 127.30.148.131:56657)
0619 16:54:49.982026 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:49.982064 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:49.982501 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:49.982708 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:49.982711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:49.982719 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:49.983182 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:49.983186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.983783 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.983784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:49.984968 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:49.984971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:49.985390 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:49.985391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:49.985524 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:49.986043 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:49.986043 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:49.986069 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:49.986089 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:49.986247 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:49.986251 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:49.986254 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:49.986317 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 16:54:49.986388 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260619 16:54:49.986975 8518 heartbeater.cc:347] Connected to a master server at 127.30.148.190:36017
I20260619 16:54:49.987056 8518 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:49.987183 8518 heartbeater.cc:511] Master 127.30.148.190:36017 requested a full tablet report, sending...
I20260619 16:54:49.987496 8067 ts_manager.cc:194] Registered new tserver with Master: 98160cf473594fd9be100caf661a8a6e (127.30.148.131:46875)
I20260619 16:54:49.987896 8067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.148.131:56657
I20260619 16:54:49.995440 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 16:54:49.997179 8108 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:49.996658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57788 (local address 127.30.148.190:36017)
0619 16:54:49.996707 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:49.996708 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:49.996849 (+ 141us) server_negotiation.cc:407] Connection header received
0619 16:54:49.996920 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:49.996921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:49.996930 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:49.996992 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:49.997111 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57788: BlockingRecv error: recv got EOF from 127.0.0.1:57788 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 16:54:49.997290 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 8128
I20260619 16:54:50.001628 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 8259
I20260619 16:54:50.006029 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 8390
I20260619 16:54:50.010216 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 8037
2026-06-19T16:54:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (632 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T16:54:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:50Z Disabled control of system clock
I20260619 16:54:50.036084 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:39725
--webserver_interface=127.30.148.190
--webserver_port=0
--builtin_ntp_servers=127.30.148.148:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:39725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 16:54:50.113991 8529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:50.114178 8529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:50.114199 8529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:50.115314 8529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:50.115355 8529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:50.115366 8529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:50.115401 8529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:50.115430 8529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:50.116719 8529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:39725
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:39725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:50.116945 8529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:50.117233 8529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:50.121742 8534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:50.121819 8537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:50.121798 8535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:50.122072 8529 server_base.cc:1034] running on GCE node
I20260619 16:54:50.122350 8529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:50.122627 8529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:50.123761 8529 hybrid_clock.cc:648] HybridClock initialized: now 1781888090123748 us; error 29 us; skew 500 ppm
I20260619 16:54:50.124866 8529 webserver.cc:460] Webserver started at http://127.30.148.190:45075/ using document root <none> and password file <none>
I20260619 16:54:50.125078 8529 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:50.125123 8529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:50.125275 8529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:50.126292 8529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "e30c8fd4b92e42baa3bf3ed32a51b82c"
format_stamp: "Formatted at 2026-06-19 16:54:50 on dist-test-slave-xb9n"
server_key: "407d37beb132a9b1d1355fda9edd2c80"
server_key_iv: "8bc4bfa86391091fde80dda142014abc"
server_key_version: "encryptionkey@0"
I20260619 16:54:50.126627 8529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "e30c8fd4b92e42baa3bf3ed32a51b82c"
format_stamp: "Formatted at 2026-06-19 16:54:50 on dist-test-slave-xb9n"
server_key: "407d37beb132a9b1d1355fda9edd2c80"
server_key_iv: "8bc4bfa86391091fde80dda142014abc"
server_key_version: "encryptionkey@0"
I20260619 16:54:50.127833 8529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:54:50.128516 8543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:50.128669 8529 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:54:50.128772 8529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "e30c8fd4b92e42baa3bf3ed32a51b82c"
format_stamp: "Formatted at 2026-06-19 16:54:50 on dist-test-slave-xb9n"
server_key: "407d37beb132a9b1d1355fda9edd2c80"
server_key_iv: "8bc4bfa86391091fde80dda142014abc"
server_key_version: "encryptionkey@0"
I20260619 16:54:50.128839 8529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:50.132059 8529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:50.132318 8529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:50.132452 8529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:50.136016 8529 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:39725
I20260619 16:54:50.136044 8595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:39725 every 8 connection(s)
I20260619 16:54:50.136356 8529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:50.136914 8596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:50.139429 8596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c: Bootstrap starting.
I20260619 16:54:50.139873 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 8529
I20260619 16:54:50.139983 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:50.140040 8596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:50.140200 31314 external_mini_cluster.cc:1428] Setting key 6a571d949b18839bfb1f75f0b4f706aa
I20260619 16:54:50.140367 8596 log.cc:824] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:50.141048 8596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c: No bootstrap required, opened a new log
I20260619 16:54:50.142588 8600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:50.140653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.30.148.190:39725)
0619 16:54:50.140827 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:50.140837 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:50.140869 (+ 32us) server_negotiation.cc:407] Connection header received
0619 16:54:50.141326 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:50.141332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:50.141410 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:50.141552 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:50.142176 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:50.142179 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:50.142179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:50.142184 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:50.142238 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:50.142247 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:50.142335 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:50.142387 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 16:54:50.142441 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260619 16:54:50.142758 8596 raft_consensus.cc:348] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30c8fd4b92e42baa3bf3ed32a51b82c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39725 } }
I20260619 16:54:50.142917 8596 raft_consensus.cc:374] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:50.142944 8596 raft_consensus.cc:729] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30c8fd4b92e42baa3bf3ed32a51b82c, State: Initialized, Role: FOLLOWER
I20260619 16:54:50.143083 8596 consensus_queue.cc:260] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30c8fd4b92e42baa3bf3ed32a51b82c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39725 } }
I20260619 16:54:50.143163 8596 raft_consensus.cc:388] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:50.143203 8596 raft_consensus.cc:482] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:50.143250 8596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:50.144016 8596 raft_consensus.cc:504] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30c8fd4b92e42baa3bf3ed32a51b82c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39725 } }
I20260619 16:54:50.144124 8596 leader_election.cc:302] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e30c8fd4b92e42baa3bf3ed32a51b82c; no voters:
I20260619 16:54:50.144279 8596 leader_election.cc:288] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:50.144311 8601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:50.144477 8601 raft_consensus.cc:686] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [term 1 LEADER]: Becoming Leader. State: Replica: e30c8fd4b92e42baa3bf3ed32a51b82c, State: Running, Role: LEADER
I20260619 16:54:50.144546 8596 sys_catalog.cc:564] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:50.144578 8601 consensus_queue.cc:237] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30c8fd4b92e42baa3bf3ed32a51b82c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39725 } }
I20260619 16:54:50.144912 8603 sys_catalog.cc:455] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e30c8fd4b92e42baa3bf3ed32a51b82c. Latest consensus state: current_term: 1 leader_uuid: "e30c8fd4b92e42baa3bf3ed32a51b82c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30c8fd4b92e42baa3bf3ed32a51b82c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39725 } } }
I20260619 16:54:50.144939 8602 sys_catalog.cc:455] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e30c8fd4b92e42baa3bf3ed32a51b82c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30c8fd4b92e42baa3bf3ed32a51b82c" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39725 } } }
I20260619 16:54:50.145030 8603 sys_catalog.cc:458] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:50.145038 8602 sys_catalog.cc:458] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:50.145534 8609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:50.146013 8609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:50.147485 8609 catalog_manager.cc:1347] Generated new cluster ID: 57d7112978924e56b12f003a5f6b42cd
I20260619 16:54:50.147552 8609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:54:50.168684 8609 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:50.168843 8609 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:50.173419 8609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e30c8fd4b92e42baa3bf3ed32a51b82c: Generated new TSK 0
I20260619 16:54:50.173597 8609 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:54:50.178993 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39725
--builtin_ntp_servers=127.30.148.148:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 16:54:50.283531 8620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:50.283715 8620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:50.283735 8620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:50.284869 8620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:50.284907 8620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:50.284932 8620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:50.286283 8620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:50.286507 8620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:50.286782 8620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:50.292214 8625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:50.292229 8626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:50.292397 8620 server_base.cc:1034] running on GCE node
W20260619 16:54:50.292223 8628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:50.292704 8620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:50.292964 8620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:50.294134 8620 hybrid_clock.cc:648] HybridClock initialized: now 1781888090294122 us; error 27 us; skew 500 ppm
I20260619 16:54:50.295452 8620 webserver.cc:460] Webserver started at http://127.30.148.129:41805/ using document root <none> and password file <none>
I20260619 16:54:50.295696 8620 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:50.295753 8620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:50.295889 8620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:50.297060 8620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "865eddfe674b49c98c0993452d03489d"
format_stamp: "Formatted at 2026-06-19 16:54:50 on dist-test-slave-xb9n"
server_key: "5b4bf19cabf76564465d4a7fad78cb38"
server_key_iv: "4681026a0afe3a263c3622ddded64a7d"
server_key_version: "encryptionkey@0"
I20260619 16:54:50.297339 8620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "865eddfe674b49c98c0993452d03489d"
format_stamp: "Formatted at 2026-06-19 16:54:50 on dist-test-slave-xb9n"
server_key: "5b4bf19cabf76564465d4a7fad78cb38"
server_key_iv: "4681026a0afe3a263c3622ddded64a7d"
server_key_version: "encryptionkey@0"
I20260619 16:54:50.298643 8620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:54:50.299252 8634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:50.299410 8620 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:50.299512 8620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "865eddfe674b49c98c0993452d03489d"
format_stamp: "Formatted at 2026-06-19 16:54:50 on dist-test-slave-xb9n"
server_key: "5b4bf19cabf76564465d4a7fad78cb38"
server_key_iv: "4681026a0afe3a263c3622ddded64a7d"
server_key_version: "encryptionkey@0"
I20260619 16:54:50.299592 8620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:50.302783 8620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:50.303026 8620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:50.303155 8620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:50.303376 8620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:50.303675 8620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:50.303737 8620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:50.303773 8620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:50.303802 8620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:50.309206 8620 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:41391
I20260619 16:54:50.309238 8747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:41391 every 8 connection(s)
I20260619 16:54:50.309566 8620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:50.312309 8600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:50.310269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:40643 (local address 127.30.148.190:39725)
0619 16:54:50.310328 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:50.310330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:50.310572 (+ 242us) server_negotiation.cc:407] Connection header received
0619 16:54:50.310990 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:50.310991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:50.310997 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:50.311048 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:50.312148 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:50.312149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:50.312149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:50.312152 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:50.312161 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:50.312164 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:50.312202 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:50.312249 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 16:54:50.312259 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 16:54:50.312510 8750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:50.310268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:39725 (local address 127.30.148.129:40643)
0619 16:54:50.310437 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:50.310466 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:50.310901 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:50.311112 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:50.311115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:50.311123 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:50.312026 (+ 903us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:50.312027 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:50.312052 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:50.312076 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:50.312203 (+ 127us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:50.312205 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:50.312207 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:54:50.312255 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 16:54:50.312296 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260619 16:54:50.312970 8748 heartbeater.cc:347] Connected to a master server at 127.30.148.190:39725
I20260619 16:54:50.313062 8748 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:50.313196 8748 heartbeater.cc:511] Master 127.30.148.190:39725 requested a full tablet report, sending...
I20260619 16:54:50.313676 8560 ts_manager.cc:194] Registered new tserver with Master: 865eddfe674b49c98c0993452d03489d (127.30.148.129:41391)
I20260619 16:54:50.313932 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 8620
I20260619 16:54:50.314042 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:50.314239 31314 external_mini_cluster.cc:1428] Setting key 7161dbb681dd4f4e6c7760558752e112
I20260619 16:54:50.315064 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39725
--builtin_ntp_servers=127.30.148.148:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 16:54:50.392359 8751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:50.392535 8751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:50.392556 8751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:50.393694 8751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:50.393731 8751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:50.393756 8751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:50.394977 8751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:50.395171 8751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:50.395380 8751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:50.400027 8757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:50.400072 8759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:50.400120 8751 server_base.cc:1034] running on GCE node
W20260619 16:54:50.400086 8756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:50.400416 8751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:50.400599 8751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:50.401723 8751 hybrid_clock.cc:648] HybridClock initialized: now 1781888090401714 us; error 26 us; skew 500 ppm
I20260619 16:54:50.402872 8751 webserver.cc:460] Webserver started at http://127.30.148.130:37435/ using document root <none> and password file <none>
I20260619 16:54:50.403105 8751 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:50.403151 8751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:50.403239 8751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:50.404230 8751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "346c91a69afe43b8ae2f9f068317439a"
format_stamp: "Formatted at 2026-06-19 16:54:50 on dist-test-slave-xb9n"
server_key: "5f9385c7c50a1747fab22cc9c6ad652e"
server_key_iv: "bbd6e46fda58df656eba41436fcd5510"
server_key_version: "encryptionkey@0"
I20260619 16:54:50.404565 8751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "346c91a69afe43b8ae2f9f068317439a"
format_stamp: "Formatted at 2026-06-19 16:54:50 on dist-test-slave-xb9n"
server_key: "5f9385c7c50a1747fab22cc9c6ad652e"
server_key_iv: "bbd6e46fda58df656eba41436fcd5510"
server_key_version: "encryptionkey@0"
I20260619 16:54:50.405853 8751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:50.406489 8765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:50.406656 8751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:50.406759 8751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "346c91a69afe43b8ae2f9f068317439a"
format_stamp: "Formatted at 2026-06-19 16:54:50 on dist-test-slave-xb9n"
server_key: "5f9385c7c50a1747fab22cc9c6ad652e"
server_key_iv: "bbd6e46fda58df656eba41436fcd5510"
server_key_version: "encryptionkey@0"
I20260619 16:54:50.406847 8751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:50.410099 8751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:50.410333 8751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:50.410459 8751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:50.410650 8751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:50.410930 8751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:50.410979 8751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:50.411017 8751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:50.411044 8751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:50.416273 8751 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:33555
I20260619 16:54:50.416306 8878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:33555 every 8 connection(s)
I20260619 16:54:50.416585 8751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:54:50.419080 8600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:50.417172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:44215 (local address 127.30.148.190:39725)
0619 16:54:50.417215 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:50.417217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:50.417421 (+ 204us) server_negotiation.cc:407] Connection header received
0619 16:54:50.417809 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:50.417810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:50.417815 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:50.417868 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:50.418914 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:50.418915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:50.418915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:50.418918 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:50.418927 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:50.418930 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:50.418968 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:50.419019 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 16:54:50.419030 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:54:50.419250 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:50.417149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:39725 (local address 127.30.148.130:44215)
0619 16:54:50.417308 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:50.417343 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:50.417736 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:50.417911 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:50.417914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:50.417921 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:50.418779 (+ 858us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:50.418779 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:50.418803 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:50.418846 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:50.418968 (+ 122us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:50.418970 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:50.418972 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:54:50.419025 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 16:54:50.419066 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
I20260619 16:54:50.419337 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 8751
I20260619 16:54:50.419467 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:50.419673 31314 external_mini_cluster.cc:1428] Setting key 75b9afedef203d6dd09806e3ec874f04
I20260619 16:54:50.419745 8879 heartbeater.cc:347] Connected to a master server at 127.30.148.190:39725
I20260619 16:54:50.419836 8879 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:50.419986 8879 heartbeater.cc:511] Master 127.30.148.190:39725 requested a full tablet report, sending...
I20260619 16:54:50.420356 8560 ts_manager.cc:194] Registered new tserver with Master: 346c91a69afe43b8ae2f9f068317439a (127.30.148.130:33555)
I20260619 16:54:50.420588 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39725
--builtin_ntp_servers=127.30.148.148:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 16:54:50.498358 8882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:50.498538 8882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:50.498559 8882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:50.499624 8882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:50.499660 8882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:50.499706 8882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:50.501027 8882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:50.501230 8882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:50.501534 8882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:50.506259 8887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:50.506245 8882 server_base.cc:1034] running on GCE node
W20260619 16:54:50.506242 8888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:50.506211 8890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:50.506640 8882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:50.506870 8882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:50.508020 8882 hybrid_clock.cc:648] HybridClock initialized: now 1781888090508002 us; error 31 us; skew 500 ppm
I20260619 16:54:50.509260 8882 webserver.cc:460] Webserver started at http://127.30.148.131:34043/ using document root <none> and password file <none>
I20260619 16:54:50.509554 8882 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:50.509619 8882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:50.509770 8882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:50.510803 8882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "062f42a8b320414589913f8485e09cd4"
format_stamp: "Formatted at 2026-06-19 16:54:50 on dist-test-slave-xb9n"
server_key: "0f1bf17d88982c9c8a0daa53adeb234c"
server_key_iv: "51f06ed56ce6112de41f5425813dcb2c"
server_key_version: "encryptionkey@0"
I20260619 16:54:50.511143 8882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "062f42a8b320414589913f8485e09cd4"
format_stamp: "Formatted at 2026-06-19 16:54:50 on dist-test-slave-xb9n"
server_key: "0f1bf17d88982c9c8a0daa53adeb234c"
server_key_iv: "51f06ed56ce6112de41f5425813dcb2c"
server_key_version: "encryptionkey@0"
I20260619 16:54:50.512336 8882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:54:50.512993 8896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:50.513150 8882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:50.513264 8882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "062f42a8b320414589913f8485e09cd4"
format_stamp: "Formatted at 2026-06-19 16:54:50 on dist-test-slave-xb9n"
server_key: "0f1bf17d88982c9c8a0daa53adeb234c"
server_key_iv: "51f06ed56ce6112de41f5425813dcb2c"
server_key_version: "encryptionkey@0"
I20260619 16:54:50.513339 8882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:50.516525 8882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:50.516798 8882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:50.516942 8882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:50.517150 8882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:50.517503 8882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:50.517544 8882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:50.517575 8882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:50.517596 8882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:50.522912 8882 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:43647
I20260619 16:54:50.522943 9009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:43647 every 8 connection(s)
I20260619 16:54:50.523250 8882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:54:50.524660 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 8882
I20260619 16:54:50.524745 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:50.524919 31314 external_mini_cluster.cc:1428] Setting key 2531db57a2b206b6a027807987c10966
I20260619 16:54:50.526163 8600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:50.523904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:39687 (local address 127.30.148.190:39725)
0619 16:54:50.523953 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:50.523956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:50.524174 (+ 218us) server_negotiation.cc:407] Connection header received
0619 16:54:50.524563 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:50.524564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:50.524571 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:50.524639 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:50.525988 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:50.525989 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:50.525989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:54:50.525992 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:50.526007 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:54:50.526011 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:50.526053 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:50.526104 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 16:54:50.526115 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:54:50.526355 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:50.523898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:39725 (local address 127.30.148.131:39687)
0619 16:54:50.524073 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:50.524100 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:50.524462 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:50.524706 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:50.524709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:50.524717 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:50.525844 (+ 1127us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:54:50.525845 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:50.525878 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:54:50.525908 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:50.526059 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:50.526061 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:50.526063 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:54:50.526111 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 16:54:50.526152 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260619 16:54:50.526772 9010 heartbeater.cc:347] Connected to a master server at 127.30.148.190:39725
I20260619 16:54:50.526855 9010 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:50.526949 9010 heartbeater.cc:511] Master 127.30.148.190:39725 requested a full tablet report, sending...
I20260619 16:54:50.527273 8560 ts_manager.cc:194] Registered new tserver with Master: 062f42a8b320414589913f8485e09cd4 (127.30.148.131:43647)
I20260619 16:54:50.536625 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 16:54:50.538300 8600 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:54:50.537912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60740 (local address 127.30.148.190:39725)
0619 16:54:50.537984 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:50.537986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:50.538048 (+ 62us) server_negotiation.cc:407] Connection header received
0619 16:54:50.538082 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:50.538083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:50.538086 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:50.538130 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:50.538243 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60740: BlockingRecv error: recv got EOF from 127.0.0.1:60740 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 16:54:50.538353 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 8620
I20260619 16:54:50.543160 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 8751
I20260619 16:54:50.547230 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 8882
I20260619 16:54:50.552037 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 8529
2026-06-19T16:54:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (541 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:50 dist-test-slave-xb9n krb5kdc[9020](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:50 dist-test-slave-xb9n krb5kdc[9020](info): set up 2 sockets
Jun 19 16:54:50 dist-test-slave-xb9n krb5kdc[9020](info): commencing operation
krb5kdc: starting...
W20260619 16:54:51.586140 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:51 dist-test-slave-xb9n krb5kdc[9020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:54:51.694321 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:41221
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:41221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:51.771592 9033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:51.771803 9033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:51.771839 9033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:51.772930 9033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:54:51.772989 9033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:51.773015 9033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:51.773044 9033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:54:51.773067 9033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:54:51.774559 9033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:41221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:41221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:51.774822 9033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:51.775036 9033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:51.779656 9041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:51.779837 9033 server_base.cc:1034] running on GCE node
W20260619 16:54:51.779656 9039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:51.779649 9038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:51.780422 9033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:51.780699 9033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:51.781857 9033 hybrid_clock.cc:648] HybridClock initialized: now 1781888091781842 us; error 28 us; skew 500 ppm
Jun 19 16:54:51 dist-test-slave-xb9n krb5kdc[9020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888091, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:51.784112 9033 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:54:51.784547 9033 webserver.cc:460] Webserver started at http://127.30.148.190:46459/ using document root <none> and password file <none>
I20260619 16:54:51.784760 9033 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:51.784825 9033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:51.784968 9033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:51.786049 9033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "effbf648729c47f4b60e4471513ecbaa"
format_stamp: "Formatted at 2026-06-19 16:54:51 on dist-test-slave-xb9n"
server_key: "e65d2f8d1a16eb0719d301af0b35c531"
server_key_iv: "35fad2cbe8650c37e23368ff3bc7593a"
server_key_version: "encryptionkey@0"
I20260619 16:54:51.786419 9033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "effbf648729c47f4b60e4471513ecbaa"
format_stamp: "Formatted at 2026-06-19 16:54:51 on dist-test-slave-xb9n"
server_key: "e65d2f8d1a16eb0719d301af0b35c531"
server_key_iv: "35fad2cbe8650c37e23368ff3bc7593a"
server_key_version: "encryptionkey@0"
I20260619 16:54:51.787673 9033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:51.788369 9048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:51.788534 9033 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:51.788636 9033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "effbf648729c47f4b60e4471513ecbaa"
format_stamp: "Formatted at 2026-06-19 16:54:51 on dist-test-slave-xb9n"
server_key: "e65d2f8d1a16eb0719d301af0b35c531"
server_key_iv: "35fad2cbe8650c37e23368ff3bc7593a"
server_key_version: "encryptionkey@0"
I20260619 16:54:51.788722 9033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:51.798291 9033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:51.799963 9033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:51.800117 9033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:51.803973 9033 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:41221
I20260619 16:54:51.804011 9100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:41221 every 8 connection(s)
I20260619 16:54:51.804332 9033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:54:51.805171 9101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:51.807313 9101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa: Bootstrap starting.
I20260619 16:54:51.807866 9101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:51.808157 9101 log.cc:824] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:51.808183 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 9033
I20260619 16:54:51.808291 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:54:51.808496 31314 external_mini_cluster.cc:1428] Setting key cc7705a7303cc12d33f92b85211fef1b
I20260619 16:54:51.808856 9101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa: No bootstrap required, opened a new log
I20260619 16:54:51.811108 9101 raft_consensus.cc:348] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "effbf648729c47f4b60e4471513ecbaa" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41221 } }
I20260619 16:54:51.811251 9101 raft_consensus.cc:374] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:51.811295 9101 raft_consensus.cc:729] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: effbf648729c47f4b60e4471513ecbaa, State: Initialized, Role: FOLLOWER
I20260619 16:54:51.811434 9101 consensus_queue.cc:260] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "effbf648729c47f4b60e4471513ecbaa" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41221 } }
I20260619 16:54:51.811528 9101 raft_consensus.cc:388] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:54:51.811569 9101 raft_consensus.cc:482] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:54:51.811614 9101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:51.812093 9101 raft_consensus.cc:504] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "effbf648729c47f4b60e4471513ecbaa" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41221 } }
I20260619 16:54:51.812203 9101 leader_election.cc:302] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: effbf648729c47f4b60e4471513ecbaa; no voters:
I20260619 16:54:51.812359 9101 leader_election.cc:288] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:54:51.812383 9106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:51.812477 9106 raft_consensus.cc:686] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [term 1 LEADER]: Becoming Leader. State: Replica: effbf648729c47f4b60e4471513ecbaa, State: Running, Role: LEADER
I20260619 16:54:51.812587 9101 sys_catalog.cc:564] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:54:51.812558 9106 consensus_queue.cc:237] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "effbf648729c47f4b60e4471513ecbaa" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41221 } }
I20260619 16:54:51.812875 9106 sys_catalog.cc:455] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "effbf648729c47f4b60e4471513ecbaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "effbf648729c47f4b60e4471513ecbaa" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41221 } } }
I20260619 16:54:51.812944 9106 sys_catalog.cc:458] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:51.813066 9106 sys_catalog.cc:455] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader effbf648729c47f4b60e4471513ecbaa. Latest consensus state: current_term: 1 leader_uuid: "effbf648729c47f4b60e4471513ecbaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "effbf648729c47f4b60e4471513ecbaa" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41221 } } }
I20260619 16:54:51.813131 9106 sys_catalog.cc:458] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa [sys.catalog]: This master's current role is: LEADER
I20260619 16:54:51.813308 9111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:54:51.813809 9111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:54:51.815718 9111 catalog_manager.cc:1347] Generated new cluster ID: 8b0af7b0aa9b4ccbafe58d211a783a45
I20260619 16:54:51.815773 9111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:54:51 dist-test-slave-xb9n krb5kdc[9020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:51.819846 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:51.808993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49094 (local address 127.30.148.190:41221)
0619 16:54:51.809141 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:51.809149 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:51.809210 (+ 61us) server_negotiation.cc:407] Connection header received
0619 16:54:51.810089 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:51.810095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:51.810145 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:51.810257 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:51.811013 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:51.812227 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:51.813291 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:51.813925 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:51.814008 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:51.816664 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:51.816668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:51.816668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:51.816678 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:51.818372 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:51.818752 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:51.818755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:51.818756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:51.818850 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:51.819055 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:51.819056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:51.819056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:51.819268 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:51.819423 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:51.819540 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 16:54:51.819667 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260619 16:54:51.855728 9111 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:54:51.856405 9111 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:54:51.860554 9111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa: Generated new TSK 0
I20260619 16:54:51.860711 9111 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:54:51.883971 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41221
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:51.960839 9127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:51.961032 9127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:51.961054 9127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:51.962162 9127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:51.962200 9127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:51.962248 9127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:54:51.963603 9127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:51.963817 9127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:51.964056 9127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:51.968758 9133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:51.968748 9132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:51.968732 9135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:51.969040 9127 server_base.cc:1034] running on GCE node
I20260619 16:54:51.969216 9127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:51.969449 9127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:51.970594 9127 hybrid_clock.cc:648] HybridClock initialized: now 1781888091970587 us; error 31 us; skew 500 ppm
Jun 19 16:54:51 dist-test-slave-xb9n krb5kdc[9020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888091, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:51.973001 9127 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:54:51.973493 9127 webserver.cc:460] Webserver started at http://127.30.148.129:33963/ using document root <none> and password file <none>
I20260619 16:54:51.973661 9127 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:51.973698 9127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:51.973783 9127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:51.974691 9127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "8e2bd23d8123463d847727904e01b44f"
format_stamp: "Formatted at 2026-06-19 16:54:51 on dist-test-slave-xb9n"
server_key: "4ae4b62dd4de87ca0e06edb14ab5cca8"
server_key_iv: "3bfcb727623bf2265f31511041536344"
server_key_version: "encryptionkey@0"
I20260619 16:54:51.975010 9127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "8e2bd23d8123463d847727904e01b44f"
format_stamp: "Formatted at 2026-06-19 16:54:51 on dist-test-slave-xb9n"
server_key: "4ae4b62dd4de87ca0e06edb14ab5cca8"
server_key_iv: "3bfcb727623bf2265f31511041536344"
server_key_version: "encryptionkey@0"
I20260619 16:54:51.976202 9127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 16:54:51.976847 9142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:51.977000 9127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:51.977066 9127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "8e2bd23d8123463d847727904e01b44f"
format_stamp: "Formatted at 2026-06-19 16:54:51 on dist-test-slave-xb9n"
server_key: "4ae4b62dd4de87ca0e06edb14ab5cca8"
server_key_iv: "3bfcb727623bf2265f31511041536344"
server_key_version: "encryptionkey@0"
I20260619 16:54:51.977114 9127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:51.997275 9127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:51.999526 9127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:51.999641 9127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:51.999811 9127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:52.000041 9127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:52.000070 9127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:52.000090 9127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:52.000103 9127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:52.005580 9127 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:39073
I20260619 16:54:52.005630 9255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:39073 every 8 connection(s)
I20260619 16:54:52.005903 9127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:54:52.008373 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 9127
I20260619 16:54:52.008455 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:54:52.008661 31314 external_mini_cluster.cc:1428] Setting key 60ce9c07fef4ade0242cc79b609fe682
Jun 19 16:54:52 dist-test-slave-xb9n krb5kdc[9020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888091, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:52.015654 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.006531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:42063 (local address 127.30.148.190:41221)
0619 16:54:52.006570 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:52.006572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:52.006783 (+ 211us) server_negotiation.cc:407] Connection header received
0619 16:54:52.007216 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:52.007217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:52.007226 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:52.007279 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:52.008106 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.008663 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.010151 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.010450 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.010515 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.012753 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:52.012754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:52.012755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:52.012764 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:52.014284 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:52.014692 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:52.014693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:52.014694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:52.014752 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:52.014991 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:52.014992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:52.014992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:52.015106 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:52.015174 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:52.015462 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 16:54:52.015563 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 16:54:52.015792 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.006497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41221 (local address 127.30.148.129:42063)
0619 16:54:52.006667 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:52.006704 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:52.007141 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:52.007327 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:52.007330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:52.007861 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:52.008034 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.008038 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.008771 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.008773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.010052 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.010055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.010540 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.010541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.010688 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.011133 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:52.011141 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:52.012661 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:52.014392 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:52.014399 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:52.014401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:52.014617 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:52.014817 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:52.014818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:52.014818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:52.014927 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:52.015176 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:52.015183 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:52.015330 (+ 147us) client_negotiation.cc:769] Sending connection context
0619 16:54:52.015430 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 16:54:52.015577 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260619 16:54:52.016248 9256 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41221
I20260619 16:54:52.016352 9256 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:52.016551 9256 heartbeater.cc:511] Master 127.30.148.190:41221 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
I20260619 16:54:52.017074 9065 ts_manager.cc:194] Registered new tserver with Master: 8e2bd23d8123463d847727904e01b44f (127.30.148.129:39073)
I20260619 16:54:52.017750 9065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:42063
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:54:52.032447 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41221
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:52.110369 9261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:52.110555 9261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:52.110579 9261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:52.111620 9261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:52.111653 9261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:52.111678 9261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:54:52.113143 9261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:52.113313 9261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:52.113536 9261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:52.118424 9267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:52.118422 9266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:52.118422 9269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:52.118439 9261 server_base.cc:1034] running on GCE node
I20260619 16:54:52.118772 9261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:52.118978 9261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:52.120132 9261 hybrid_clock.cc:648] HybridClock initialized: now 1781888092120120 us; error 25 us; skew 500 ppm
Jun 19 16:54:52 dist-test-slave-xb9n krb5kdc[9020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888092, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:52.122478 9261 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:54:52.122918 9261 webserver.cc:460] Webserver started at http://127.30.148.130:38093/ using document root <none> and password file <none>
I20260619 16:54:52.123136 9261 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:52.123200 9261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:52.123345 9261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:52.124297 9261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "06d40532670e4e04a65e9e80f1395fb3"
format_stamp: "Formatted at 2026-06-19 16:54:52 on dist-test-slave-xb9n"
server_key: "39c39828229846d56fa31e54b9d4b5ec"
server_key_iv: "a750a3831033bdf72ffe763121c7bba3"
server_key_version: "encryptionkey@0"
I20260619 16:54:52.124639 9261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "06d40532670e4e04a65e9e80f1395fb3"
format_stamp: "Formatted at 2026-06-19 16:54:52 on dist-test-slave-xb9n"
server_key: "39c39828229846d56fa31e54b9d4b5ec"
server_key_iv: "a750a3831033bdf72ffe763121c7bba3"
server_key_version: "encryptionkey@0"
I20260619 16:54:52.126387 9261 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 16:54:52.127045 9276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:52.127194 9261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:52.127287 9261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "06d40532670e4e04a65e9e80f1395fb3"
format_stamp: "Formatted at 2026-06-19 16:54:52 on dist-test-slave-xb9n"
server_key: "39c39828229846d56fa31e54b9d4b5ec"
server_key_iv: "a750a3831033bdf72ffe763121c7bba3"
server_key_version: "encryptionkey@0"
I20260619 16:54:52.127368 9261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:52.137497 9261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:52.139061 9261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:52.139205 9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:52.139396 9261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:52.139663 9261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:52.139714 9261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:52.139748 9261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:52.139775 9261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:52.145323 9261 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:34137
I20260619 16:54:52.145417 9389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:34137 every 8 connection(s)
I20260619 16:54:52.145732 9261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:54:52.146293 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 9261
I20260619 16:54:52.146399 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:54:52.146610 31314 external_mini_cluster.cc:1428] Setting key 13e9b20208b26cff4589347e93fe9fc6
Jun 19 16:54:52 dist-test-slave-xb9n krb5kdc[9020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888092, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
I20260619 16:54:52.155527 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.146380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:60887 (local address 127.30.148.190:41221)
0619 16:54:52.146434 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:52.146437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:52.146651 (+ 214us) server_negotiation.cc:407] Connection header received
0619 16:54:52.147265 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:52.147266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:52.147277 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:52.147327 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:52.148214 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.149021 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.150056 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.150396 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.150477 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.152545 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:52.152547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:52.152547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:52.152554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:52.154201 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:52.154562 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:52.154563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:52.154564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:52.154622 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:52.154904 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:52.154905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:52.154906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:52.155021 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:52.155102 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:52.155360 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 16:54:52.155441 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 16:54:52.155669 9392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.146310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41221 (local address 127.30.148.130:60887)
0619 16:54:52.146528 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:52.146574 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:52.147185 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:52.147408 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:52.147412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:52.147950 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:52.148119 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.148123 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.149119 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.149120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.149962 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.149969 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.150539 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.150540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.150638 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.151138 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:52.151146 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:52.152456 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:52.154283 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:52.154288 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:52.154290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:52.154488 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:52.154695 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:52.154696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:52.154696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:52.154822 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:52.155093 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:52.155099 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:52.155230 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 16:54:52.155328 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 16:54:52.155466 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":82,"threads_started":1}
I20260619 16:54:52.156241 9390 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41221
I20260619 16:54:52.156328 9390 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:52.156447 9390 heartbeater.cc:511] Master 127.30.148.190:41221 requested a full tablet report, sending...
I20260619 16:54:52.156808 9065 ts_manager.cc:194] Registered new tserver with Master: 06d40532670e4e04a65e9e80f1395fb3 (127.30.148.130:34137)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
I20260619 16:54:52.157193 9065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:60887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:54:52.170401 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41221
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:54:52.248301 9395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:54:52.248483 9395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:54:52.248513 9395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:54:52.249640 9395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:54:52.249678 9395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:54:52.249704 9395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:54:52.250998 9395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:54:52.251157 9395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:54:52.251309 9395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:54:52.255931 9401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:54:52.255982 9400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:52.255970 9395 server_base.cc:1034] running on GCE node
W20260619 16:54:52.255944 9403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:54:52.256350 9395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:54:52.256633 9395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:54:52.257768 9395 hybrid_clock.cc:648] HybridClock initialized: now 1781888092257759 us; error 31 us; skew 500 ppm
Jun 19 16:54:52 dist-test-slave-xb9n krb5kdc[9020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888092, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:54:52.260208 9395 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:54:52.260569 9395 webserver.cc:460] Webserver started at http://127.30.148.131:34751/ using document root <none> and password file <none>
I20260619 16:54:52.260738 9395 fs_manager.cc:359] Metadata directory not provided
I20260619 16:54:52.260777 9395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:54:52.260862 9395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:54:52.261848 9395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "02e4301e4b824ef6901d5801623a4b75"
format_stamp: "Formatted at 2026-06-19 16:54:52 on dist-test-slave-xb9n"
server_key: "bf5f860efa1b48b611206f71ad565081"
server_key_iv: "dfe57a7a73046ba3645cb9225d3a7af0"
server_key_version: "encryptionkey@0"
I20260619 16:54:52.262136 9395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "02e4301e4b824ef6901d5801623a4b75"
format_stamp: "Formatted at 2026-06-19 16:54:52 on dist-test-slave-xb9n"
server_key: "bf5f860efa1b48b611206f71ad565081"
server_key_iv: "dfe57a7a73046ba3645cb9225d3a7af0"
server_key_version: "encryptionkey@0"
I20260619 16:54:52.263305 9395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:54:52.263913 9410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:52.264123 9395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:54:52.264204 9395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "02e4301e4b824ef6901d5801623a4b75"
format_stamp: "Formatted at 2026-06-19 16:54:52 on dist-test-slave-xb9n"
server_key: "bf5f860efa1b48b611206f71ad565081"
server_key_iv: "dfe57a7a73046ba3645cb9225d3a7af0"
server_key_version: "encryptionkey@0"
I20260619 16:54:52.264254 9395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:54:52.274437 9395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:54:52.276561 9395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:54:52.276664 9395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:54:52.276818 9395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:54:52.277034 9395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:54:52.277064 9395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:52.277083 9395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:54:52.277096 9395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:54:52.282572 9395 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:44273
I20260619 16:54:52.282648 9523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:44273 every 8 connection(s)
I20260619 16:54:52.282919 9395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:54:52.284387 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 9395
I20260619 16:54:52.284498 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:54:52.284705 31314 external_mini_cluster.cc:1428] Setting key 9575ac24d031629c3b0a455b877c7aab
Jun 19 16:54:52 dist-test-slave-xb9n krb5kdc[9020](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888092, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:54:52.291975 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.283642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:55223 (local address 127.30.148.190:41221)
0619 16:54:52.283687 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:52.283689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:52.283909 (+ 220us) server_negotiation.cc:407] Connection header received
0619 16:54:52.284521 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:52.284522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:52.284535 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:52.284596 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:52.285683 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.286162 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.287077 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.287261 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.287310 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.289229 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:52.289230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:52.289230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:52.289237 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:52.290747 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:52.291099 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:52.291100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:52.291101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:52.291151 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:52.291328 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:52.291328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:52.291329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:52.291423 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:52.291473 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:52.291845 (+ 372us) server_negotiation.cc:299] Negotiation successful
0619 16:54:52.291914 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:54:52.292104 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.283629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41221 (local address 127.30.148.131:55223)
0619 16:54:52.283802 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:52.283833 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:52.284400 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:52.284682 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:52.284686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:52.285352 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:54:52.285594 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.285598 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.286237 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.286237 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.286986 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.286988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.287327 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.287328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.287420 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.287830 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:54:52.287838 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:54:52.289146 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:54:52.290832 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:52.290838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:52.290839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:52.291040 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:52.291204 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:54:52.291205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:54:52.291205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:54:52.291280 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:54:52.291476 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:54:52.291480 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:54:52.291581 (+ 101us) client_negotiation.cc:769] Sending connection context
0619 16:54:52.291811 (+ 230us) client_negotiation.cc:240] Negotiation successful
0619 16:54:52.291921 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
I20260619 16:54:52.292548 9524 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41221
I20260619 16:54:52.292652 9524 heartbeater.cc:464] Registering TS with master...
I20260619 16:54:52.292810 9524 heartbeater.cc:511] Master 127.30.148.190:41221 requested a full tablet report, sending...
I20260619 16:54:52.293107 9065 ts_manager.cc:194] Registered new tserver with Master: 02e4301e4b824ef6901d5801623a4b75 (127.30.148.131:44273)
I20260619 16:54:52.293556 9065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:55223
I20260619 16:54:52.296334 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:54:52.303054 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.297553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.30.148.190:41221)
0619 16:54:52.297602 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:52.297604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:52.297705 (+ 101us) server_negotiation.cc:407] Connection header received
0619 16:54:52.297746 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:52.297746 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:52.297754 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:52.297799 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:54:52.298283 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.298720 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.299382 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.299551 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.299603 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.300623 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:54:52.300624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:54:52.300624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:54:52.300629 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:54:52.301881 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:52.302234 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:52.302235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:52.302236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:52.302300 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:54:52.302494 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:54:52.302495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:54:52.302495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:54:52.302623 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:54:52.302725 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:52.302877 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 16:54:52.302952 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 16:54:52.304335 9065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:54:52.305884 9065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:54:52.311955 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.308044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:39073 (local address 127.0.0.1:54830)
0619 16:54:52.308283 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:52.308294 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:52.308348 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:52.308660 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:52.308661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:52.308664 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:52.308714 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.308716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.309633 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.309634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.310811 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.310813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.311640 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.311641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.311724 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.311737 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 16:54:52.311811 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 16:54:52.311859 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":84,"threads_started":1}
I20260619 16:54:52.311971 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.308279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.30.148.129:39073)
0619 16:54:52.308431 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:52.308434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:52.308442 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:52.308472 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:52.308473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:52.308485 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:52.308579 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:52.308796 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.309539 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.310896 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.311547 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.311629 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.311680 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:52.311840 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 16:54:52.311892 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260619 16:54:52.312143 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.307825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:34137 (local address 127.0.0.1:41172)
0619 16:54:52.308108 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:52.308148 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:52.308225 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:52.308619 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:52.308622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:52.308623 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:52.308684 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.308686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.310047 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.310048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.311027 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.311029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.311922 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.311923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.311990 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.311995 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:54:52.312028 (+ 33us) client_negotiation.cc:240] Negotiation successful
0619 16:54:52.312075 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":77,"threads_started":1}
I20260619 16:54:52.312443 9534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.307920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.30.148.130:34137)
0619 16:54:52.308114 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:52.308118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:52.308432 (+ 314us) server_negotiation.cc:407] Connection header received
0619 16:54:52.308484 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:52.308487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:52.308503 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:52.308761 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:52.308835 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.309930 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.311101 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.311825 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.312214 (+ 389us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.312275 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:52.312303 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 16:54:52.312347 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260619 16:54:52.313112 9324 tablet_service.cc:1467] Processing CreateTablet for tablet 6967bf02f0e649eda6733821ac31d10b (DEFAULT_TABLE table=test-table [id=a8d50c4de44341b1b4da142af7c6883a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:52.313146 9190 tablet_service.cc:1467] Processing CreateTablet for tablet 6967bf02f0e649eda6733821ac31d10b (DEFAULT_TABLE table=test-table [id=a8d50c4de44341b1b4da142af7c6883a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:52.313397 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.307868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:44273 (local address 127.0.0.1:43814)
0619 16:54:52.308110 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:52.308148 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:52.308213 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:52.308716 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:52.308716 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:52.308719 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:52.308754 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.308755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.311095 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.311096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.311849 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.311851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.313128 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.313130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.313207 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.313211 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:54:52.313262 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 16:54:52.313301 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":58,"threads_started":1}
I20260619 16:54:52.313463 9324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6967bf02f0e649eda6733821ac31d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:52.313509 9190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6967bf02f0e649eda6733821ac31d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:52.313599 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.307989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43814 (local address 127.30.148.131:44273)
0619 16:54:52.308459 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:52.308463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:52.308472 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:54:52.308534 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:52.308536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:52.308549 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:52.308620 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:52.308866 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.309937 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.311932 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.312946 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.313045 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.313094 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:52.313489 (+ 395us) server_negotiation.cc:299] Negotiation successful
0619 16:54:52.313531 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":64,"threads_started":1}
I20260619 16:54:52.316494 9538 tablet_bootstrap.cc:492] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3: Bootstrap starting.
I20260619 16:54:52.316536 9539 tablet_bootstrap.cc:492] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f: Bootstrap starting.
I20260619 16:54:52.317152 9538 tablet_bootstrap.cc:654] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:52.317200 9539 tablet_bootstrap.cc:654] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:52.317173 9458 tablet_service.cc:1467] Processing CreateTablet for tablet 6967bf02f0e649eda6733821ac31d10b (DEFAULT_TABLE table=test-table [id=a8d50c4de44341b1b4da142af7c6883a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:54:52.317528 9458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6967bf02f0e649eda6733821ac31d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:54:52.317775 9538 log.cc:824] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:52.317795 9539 log.cc:824] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:52.318518 9538 tablet_bootstrap.cc:492] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3: No bootstrap required, opened a new log
I20260619 16:54:52.318593 9538 ts_tablet_manager.cc:1397] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:54:52.320674 9542 tablet_bootstrap.cc:492] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75: Bootstrap starting.
I20260619 16:54:52.320629 9538 raft_consensus.cc:348] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d40532670e4e04a65e9e80f1395fb3" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34137 } } peers { permanent_uuid: "8e2bd23d8123463d847727904e01b44f" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 39073 } } peers { permanent_uuid: "02e4301e4b824ef6901d5801623a4b75" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 44273 } }
I20260619 16:54:52.320787 9538 raft_consensus.cc:374] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:52.320834 9538 raft_consensus.cc:729] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d40532670e4e04a65e9e80f1395fb3, State: Initialized, Role: FOLLOWER
I20260619 16:54:52.320981 9538 consensus_queue.cc:260] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d40532670e4e04a65e9e80f1395fb3" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34137 } } peers { permanent_uuid: "8e2bd23d8123463d847727904e01b44f" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 39073 } } peers { permanent_uuid: "02e4301e4b824ef6901d5801623a4b75" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 44273 } }
I20260619 16:54:52.321269 9542 tablet_bootstrap.cc:654] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75: Neither blocks nor log segments found. Creating new log.
I20260619 16:54:52.321281 9538 ts_tablet_manager.cc:1428] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:54:52.321410 9390 heartbeater.cc:503] Master 127.30.148.190:41221 was elected leader, sending a full tablet report...
I20260619 16:54:52.321657 9539 tablet_bootstrap.cc:492] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f: No bootstrap required, opened a new log
I20260619 16:54:52.321731 9539 ts_tablet_manager.cc:1397] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260619 16:54:52.321868 9542 log.cc:824] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75: Log is configured to *not* fsync() on all Append() calls
I20260619 16:54:52.322568 9542 tablet_bootstrap.cc:492] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75: No bootstrap required, opened a new log
I20260619 16:54:52.322623 9542 ts_tablet_manager.cc:1397] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 16:54:52.323470 9539 raft_consensus.cc:348] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d40532670e4e04a65e9e80f1395fb3" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34137 } } peers { permanent_uuid: "8e2bd23d8123463d847727904e01b44f" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 39073 } } peers { permanent_uuid: "02e4301e4b824ef6901d5801623a4b75" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 44273 } }
I20260619 16:54:52.323607 9539 raft_consensus.cc:374] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:52.323652 9539 raft_consensus.cc:729] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e2bd23d8123463d847727904e01b44f, State: Initialized, Role: FOLLOWER
I20260619 16:54:52.323777 9539 consensus_queue.cc:260] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d40532670e4e04a65e9e80f1395fb3" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34137 } } peers { permanent_uuid: "8e2bd23d8123463d847727904e01b44f" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 39073 } } peers { permanent_uuid: "02e4301e4b824ef6901d5801623a4b75" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 44273 } }
I20260619 16:54:52.324049 9539 ts_tablet_manager.cc:1428] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 16:54:52.324088 9256 heartbeater.cc:503] Master 127.30.148.190:41221 was elected leader, sending a full tablet report...
I20260619 16:54:52.324501 9542 raft_consensus.cc:348] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d40532670e4e04a65e9e80f1395fb3" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34137 } } peers { permanent_uuid: "8e2bd23d8123463d847727904e01b44f" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 39073 } } peers { permanent_uuid: "02e4301e4b824ef6901d5801623a4b75" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 44273 } }
I20260619 16:54:52.324649 9542 raft_consensus.cc:374] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:54:52.324684 9542 raft_consensus.cc:729] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02e4301e4b824ef6901d5801623a4b75, State: Initialized, Role: FOLLOWER
I20260619 16:54:52.324795 9542 consensus_queue.cc:260] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d40532670e4e04a65e9e80f1395fb3" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34137 } } peers { permanent_uuid: "8e2bd23d8123463d847727904e01b44f" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 39073 } } peers { permanent_uuid: "02e4301e4b824ef6901d5801623a4b75" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 44273 } }
I20260619 16:54:52.325057 9524 heartbeater.cc:503] Master 127.30.148.190:41221 was elected leader, sending a full tablet report...
I20260619 16:54:52.325058 9542 ts_tablet_manager.cc:1428] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260619 16:54:52.396251 9391 tablet.cc:2369] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:54:52.416260 9545 raft_consensus.cc:482] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:54:52.416357 9545 raft_consensus.cc:504] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d40532670e4e04a65e9e80f1395fb3" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34137 } } peers { permanent_uuid: "8e2bd23d8123463d847727904e01b44f" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 39073 } } peers { permanent_uuid: "02e4301e4b824ef6901d5801623a4b75" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 44273 } }
I20260619 16:54:52.416659 9545 leader_election.cc:288] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d40532670e4e04a65e9e80f1395fb3 (127.30.148.130:34137), 02e4301e4b824ef6901d5801623a4b75 (127.30.148.131:44273)
I20260619 16:54:52.419497 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.416980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:34181 (local address 127.30.148.131:44273)
0619 16:54:52.417020 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:52.417021 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:52.417029 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:54:52.417060 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:52.417061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:52.417068 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:52.417146 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:52.417340 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.417858 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.418760 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.419122 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.419208 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.419217 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:52.419395 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 16:54:52.419433 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:54:52.419495 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.416848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:44273 (local address 127.30.148.129:34181)
0619 16:54:52.416912 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:52.416926 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:52.416982 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:52.417164 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:52.417165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:52.417168 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:52.417271 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.417272 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.417943 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.417944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.418659 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.418660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.419211 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.419212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.419290 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.419293 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:52.419386 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 16:54:52.419424 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 16:54:52.419588 9534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.416930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:55617 (local address 127.30.148.130:34137)
0619 16:54:52.416977 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:52.416979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:52.417195 (+ 216us) server_negotiation.cc:407] Connection header received
0619 16:54:52.417251 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:52.417252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:52.417262 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:52.417335 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:54:52.417514 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.417971 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.418871 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.419234 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.419316 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.419341 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:52.419488 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 16:54:52.419523 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:54:52.419600 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.416848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:34137 (local address 127.30.148.129:55617)
0619 16:54:52.417104 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:52.417112 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:54:52.417172 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:54:52.417339 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:54:52.417340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:54:52.417342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:54:52.417442 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.417444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.418084 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.418085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.418782 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:54:52.418784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.419315 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.419316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:54:52.419410 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.419413 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:54:52.419481 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 16:54:52.419526 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":47,"thread_start_us":122,"threads_started":1}
I20260619 16:54:52.419903 9344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6967bf02f0e649eda6733821ac31d10b" candidate_uuid: "8e2bd23d8123463d847727904e01b44f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d40532670e4e04a65e9e80f1395fb3" is_pre_election: true
I20260619 16:54:52.419932 9478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6967bf02f0e649eda6733821ac31d10b" candidate_uuid: "8e2bd23d8123463d847727904e01b44f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02e4301e4b824ef6901d5801623a4b75" is_pre_election: true
I20260619 16:54:52.420070 9344 raft_consensus.cc:2454] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e2bd23d8123463d847727904e01b44f in term 0.
I20260619 16:54:52.420080 9478 raft_consensus.cc:2454] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e2bd23d8123463d847727904e01b44f in term 0.
I20260619 16:54:52.420303 9145 leader_election.cc:302] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d40532670e4e04a65e9e80f1395fb3, 8e2bd23d8123463d847727904e01b44f; no voters:
I20260619 16:54:52.420449 9545 raft_consensus.cc:2781] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:54:52.420502 9545 raft_consensus.cc:482] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:54:52.420531 9545 raft_consensus.cc:3037] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:52.421181 9545 raft_consensus.cc:504] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d40532670e4e04a65e9e80f1395fb3" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34137 } } peers { permanent_uuid: "8e2bd23d8123463d847727904e01b44f" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 39073 } } peers { permanent_uuid: "02e4301e4b824ef6901d5801623a4b75" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 44273 } }
I20260619 16:54:52.421312 9545 leader_election.cc:288] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [CANDIDATE]: Term 1 election: Requested vote from peers 06d40532670e4e04a65e9e80f1395fb3 (127.30.148.130:34137), 02e4301e4b824ef6901d5801623a4b75 (127.30.148.131:44273)
I20260619 16:54:52.421443 9344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6967bf02f0e649eda6733821ac31d10b" candidate_uuid: "8e2bd23d8123463d847727904e01b44f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d40532670e4e04a65e9e80f1395fb3"
I20260619 16:54:52.421496 9478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6967bf02f0e649eda6733821ac31d10b" candidate_uuid: "8e2bd23d8123463d847727904e01b44f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02e4301e4b824ef6901d5801623a4b75"
I20260619 16:54:52.421526 9344 raft_consensus.cc:3037] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:52.421571 9478 raft_consensus.cc:3037] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:54:52.422219 9344 raft_consensus.cc:2454] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e2bd23d8123463d847727904e01b44f in term 1.
I20260619 16:54:52.422241 9478 raft_consensus.cc:2454] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e2bd23d8123463d847727904e01b44f in term 1.
I20260619 16:54:52.422385 9145 leader_election.cc:302] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d40532670e4e04a65e9e80f1395fb3, 8e2bd23d8123463d847727904e01b44f; no voters:
I20260619 16:54:52.422535 9545 raft_consensus.cc:2781] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:54:52.422688 9545 raft_consensus.cc:686] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [term 1 LEADER]: Becoming Leader. State: Replica: 8e2bd23d8123463d847727904e01b44f, State: Running, Role: LEADER
I20260619 16:54:52.422780 9545 consensus_queue.cc:237] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d40532670e4e04a65e9e80f1395fb3" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34137 } } peers { permanent_uuid: "8e2bd23d8123463d847727904e01b44f" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 39073 } } peers { permanent_uuid: "02e4301e4b824ef6901d5801623a4b75" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 44273 } }
I20260619 16:54:52.423545 9064 catalog_manager.cc:5495] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e2bd23d8123463d847727904e01b44f (127.30.148.129). New cstate: current_term: 1 leader_uuid: "8e2bd23d8123463d847727904e01b44f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d40532670e4e04a65e9e80f1395fb3" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e2bd23d8123463d847727904e01b44f" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 39073 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02e4301e4b824ef6901d5801623a4b75" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 44273 } health_report { overall_health: UNKNOWN } } }
I20260619 16:54:52.429795 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.427376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54838 (local address 127.30.148.129:39073)
0619 16:54:52.427436 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 16:54:52.427439 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:54:52.427457 (+ 18us) server_negotiation.cc:407] Connection header received
0619 16:54:52.427550 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:54:52.427551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:54:52.427567 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:54:52.427636 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:54:52.427787 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.428222 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.428958 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:54:52.429157 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:54:52.429258 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:54:52.429474 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:54:52.429571 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:54:52.429664 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 16:54:52.429682 (+ 18us) server_negotiation.cc:299] Negotiation successful
0619 16:54:52.429707 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 16:54:52.431473 9478 raft_consensus.cc:1261] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75 [term 1 FOLLOWER]: Refusing update from remote peer 8e2bd23d8123463d847727904e01b44f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:52.431473 9344 raft_consensus.cc:1261] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3 [term 1 FOLLOWER]: Refusing update from remote peer 8e2bd23d8123463d847727904e01b44f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:54:52.431769 9545 consensus_queue.cc:1035] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [LEADER]: Connected to new peer: Peer: permanent_uuid: "02e4301e4b824ef6901d5801623a4b75" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 44273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:52.431892 9548 consensus_queue.cc:1035] T 6967bf02f0e649eda6733821ac31d10b P 8e2bd23d8123463d847727904e01b44f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d40532670e4e04a65e9e80f1395fb3" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:54:52.436062 9065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:52.436156 9065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:54:52.436790 9065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P effbf648729c47f4b60e4471513ecbaa: Sending DeleteTablet for 3 replicas of tablet 6967bf02f0e649eda6733821ac31d10b
I20260619 16:54:52.437222 9458 tablet_service.cc:1514] Processing DeleteTablet for tablet 6967bf02f0e649eda6733821ac31d10b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:52 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:43814
I20260619 16:54:52.437348 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 9127
I20260619 16:54:52.437475 9324 tablet_service.cc:1514] Processing DeleteTablet for tablet 6967bf02f0e649eda6733821ac31d10b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:54:52 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:41172
I20260619 16:54:52.438172 9562 tablet_replica.cc:330] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3: stopping tablet replica
I20260619 16:54:52.438285 9562 raft_consensus.cc:2229] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:52.438385 9562 raft_consensus.cc:2258] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:52.438557 9561 tablet_replica.cc:330] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75: stopping tablet replica
I20260619 16:54:52.438656 9561 raft_consensus.cc:2229] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:54:52.438745 9561 raft_consensus.cc:2258] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:54:52.438781 9562 ts_tablet_manager.cc:1905] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:52.439406 9561 ts_tablet_manager.cc:1905] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:54:52.440280 9562 ts_tablet_manager.cc:1918] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:52.440361 9562 log.cc:1196] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/6967bf02f0e649eda6733821ac31d10b
I20260619 16:54:52.440657 9562 ts_tablet_manager.cc:1939] T 6967bf02f0e649eda6733821ac31d10b P 06d40532670e4e04a65e9e80f1395fb3: Deleting consensus metadata
I20260619 16:54:52.441033 9051 catalog_manager.cc:4841] TS 06d40532670e4e04a65e9e80f1395fb3 (127.30.148.130:34137): tablet 6967bf02f0e649eda6733821ac31d10b (table test-table [id=a8d50c4de44341b1b4da142af7c6883a]) successfully deleted
I20260619 16:54:52.441244 9561 ts_tablet_manager.cc:1918] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:54:52.441318 9561 log.cc:1196] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/6967bf02f0e649eda6733821ac31d10b
I20260619 16:54:52.441619 9561 ts_tablet_manager.cc:1939] T 6967bf02f0e649eda6733821ac31d10b P 02e4301e4b824ef6901d5801623a4b75: Deleting consensus metadata
I20260619 16:54:52.441995 9052 catalog_manager.cc:4841] TS 02e4301e4b824ef6901d5801623a4b75 (127.30.148.131:44273): tablet 6967bf02f0e649eda6733821ac31d10b (table test-table [id=a8d50c4de44341b1b4da142af7c6883a]) successfully deleted
W20260619 16:54:52.444142 9051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:39073 (error 108)
I20260619 16:54:52.444310 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 9261
I20260619 16:54:52.444908 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:54:52.444432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:39073 (local address 127.0.0.1:54852)
0619 16:54:52.444826 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0619 16:54:52.444860 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:39073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378}
W20260619 16:54:52.445092 9051 catalog_manager.cc:4584] TS 8e2bd23d8123463d847727904e01b44f (127.30.148.129:39073): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6967bf02f0e649eda6733821ac31d10b: Network error: Client connection negotiation failed: client connection to 127.30.148.129:39073: connect: Connection refused (error 111)
I20260619 16:54:52.448998 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 9395
I20260619 16:54:52.453074 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 9033
2026-06-19T16:54:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1901 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:54:52 dist-test-slave-xb9n krb5kdc[9566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:54:52 dist-test-slave-xb9n krb5kdc[9566](info): set up 2 sockets
Jun 19 16:54:52 dist-test-slave-xb9n krb5kdc[9566](info): commencing operation
krb5kdc: starting...
W20260619 16:54:53.482844 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:54:53 dist-test-slave-xb9n krb5kdc[9566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:54:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:54:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.148.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.148.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/rangeradmin_127.30.148.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/rangeradmin_127.30.148.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.148.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.148.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/rangerlookup_127.30.148.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/rangerlookup_127.30.148.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.148.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.148.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/HTTP_127.30.148.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/HTTP_127.30.148.148@KRBTEST.COM.keytab.
I20260619 16:54:53.632709 31314 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLxMmiv/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/postgres -l logfile start
2026-06-19 16:54:55.901 UTC [9596] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 16:54:55.901 UTC [9596] LOG: listening on IPv4 address "127.30.148.148", port 42223
2026-06-19 16:54:55.904 UTC [9596] LOG: listening on Unix socket "/tmp/.s.PGSQL.42223"
2026-06-19 16:54:55.936 UTC [9599] LOG: database system was shut down at 2026-06-19 16:54:54 UTC
2026-06-19 16:54:55.940 UTC [9596] LOG: database system is ready to accept connections
I20260619 16:54:56.902501 31314 mini_postgres.cc:96] Postgres bound to 42223
2026-06-19 16:54:56.908 UTC [9607] FATAL: database "slave" does not exist
127.30.148.148:42223 - accepting connections
I20260619 16:54:56.909003 31314 mini_ranger.cc:161] Starting Ranger...
I20260619 16:54:56.922394 31314 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 16:54:57.323020 31314 mini_ranger.cc:87] Created ranger Postgres database
I20260619 16:54:57.323103 31314 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-admin
2026-06-19 16:54:57,614 [I] DB FLAVOR :POSTGRES
2026-06-19 16:54:57,614 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 16:54:57,614 [I] Checking connection..
2026-06-19 16:54:57,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 16:54:57,837 [I] Checking connection passed.
2026-06-19 16:54:57,837 [I] --------- Verifying version history table ---------
2026-06-19 16:54:57,837 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 16:54:58,094 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 16:54:58,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 16:54:58,324 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 16:54:58,325 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 16:54:58,325 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 16:54:58.561 UTC [9671] WARNING: there is no transaction in progress
2026-06-19 16:54:58,566 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 16:54:58,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 16:54:58,809 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 16:54:58,809 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 16:54:58,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 16:54:59,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 16:54:59,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-xb9n.c.gcp-upstream.internal','N') ;"
2026-06-19 16:54:59,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 16:54:59,773 [I] Table x_portal_user does not exist in database ranger
2026-06-19 16:54:59,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 16:55:00,005 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 16:55:00,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 16:55:00,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 16:55:00,478 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 16:55:00,478 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 16:55:01.353 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.369 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.383 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.396 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.413 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.433 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.494 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.501 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.513 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.523 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.532 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.545 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.554 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.559 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.565 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:01.571 UTC [9779] WARNING: there is no transaction in progress
2026-06-19 16:55:02,127 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 16:55:02,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 16:55:02,341 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 16:55:02,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 16:55:02,595 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 16:55:02,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 16:55:02,852 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 16:55:02,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 16:55:03,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 16:55:03,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-xb9n.c.gcp-upstream.internal';"
2026-06-19 16:55:03,520 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 16:55:03,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 16:55:03,756 [I] DB_PATCHES have already been applied
I20260619 16:55:03.762521 31314 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLxMmiv/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 16:55:03.762600 31314 mini_ranger.cc:193] Using host: 127.30.148.148
I20260619 16:55:03.765466 31314 mini_ranger.cc:227] Ranger admin URL: http://127.30.148.148:40795
Jun 19, 2026 4:55:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 4:55:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 4:55:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 4:55:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 4:55:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 4:55:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 4:55:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 16:55:04.871780 31314 mini_ranger.cc:160] Time spent starting Ranger: real 7.963s user 0.000s sys 0.009s
Jun 19, 2026 4:55:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 4:55:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 19, 2026 4:55:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 4:55:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 4:55:23 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 16:55:24.463780 31314 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.148.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.148.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/rangerkms_127.30.148.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/rangerkms_127.30.148.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.148.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.148.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.148.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/HTTP_127.30.148.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.148.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/HTTP_127.30.148.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 16:55:24 dist-test-slave-xb9n krb5kdc[9566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888124, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 16:55:24.551383 31314 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 16:55:24.559845 31314 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 16:55:25.259555 31314 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 16:55:25.259644 31314 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-kms
I20260619 16:55:25.259665 31314 mini_ranger_kms.cc:227] Using postgres at 127.30.148.148:42223
2026-06-19 16:55:25,428 [I] DB FLAVOR :POSTGRES
2026-06-19 16:55:25,428 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 16:55:25,428 [I] Checking connection
2026-06-19 16:55:25,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 16:55:25,660 [I] connection success
2026-06-19 16:55:25,661 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 16:55:25,661 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 16:55:25,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 16:55:25,915 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 16:55:25,915 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 16:55:25,915 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 16:55:25,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:42223/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 16:55:26,170 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 16:55:26.600481 31314 mini_ranger_kms.cc:310] Created kms service
I20260619 16:55:26.762001 31314 mini_ranger_kms.cc:326] Created kudu user
I20260619 16:55:26.801702 31314 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 16:55:26 dist-test-slave-xb9n krb5kdc[9566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888124, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:55:27.209180 31314 mini_ranger_kms.cc:384] Added ranger policy
I20260619 16:55:27.209299 31314 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 16:55:27.209323 31314 mini_ranger_kms.cc:240] Using host: 127.30.148.148
I20260619 16:55:27.212651 31314 mini_ranger_kms.cc:276] Ranger KMS PID: 9959
I20260619 16:55:27.212761 31314 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.148.148:43721
Jun 19, 2026 4:55:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 4:55:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 4:55:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 4:55:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 4:55:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 4:55:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 19 16:55:29 dist-test-slave-xb9n krb5kdc[9566](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781888129, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.148.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:55:29.459837 31314 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.908s user 0.004s sys 0.149s
I20260619 16:55:29.459945 31314 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 16:55:29.460031 31314 mini_ranger_kms.cc:401] 127.30.148.148:43721/kms/v1/keys
Jun 19 16:55:29 dist-test-slave-xb9n krb5kdc[9566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888129, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.148.148@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
Jun 19 16:55:30 dist-test-slave-xb9n krb5kdc[9566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888129, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.148.148@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
Jun 19 16:55:32 dist-test-slave-xb9n krb5kdc[9566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:55:32.279901 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:39031
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:39031
--ranger_config_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.148.148:43721/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:55:32.361655 10010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:55:32.361829 10010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:55:32.361850 10010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:55:32.362973 10010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:55:32.363014 10010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:55:32.363029 10010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:55:32.363040 10010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:55:32.363085 10010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:55:32.364602 10010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.148.148:43721/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:39031
--ranger_config_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:39031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:55:32.364821 10010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:55:32.365044 10010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:55:32.369994 10018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:55:32.370162 10015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:55:32.370029 10010 server_base.cc:1034] running on GCE node
W20260619 16:55:32.370019 10016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:55:32.370491 10010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:55:32.370806 10010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:55:32.371954 10010 hybrid_clock.cc:648] HybridClock initialized: now 1781888132371943 us; error 29 us; skew 500 ppm
Jun 19 16:55:32 dist-test-slave-xb9n krb5kdc[9566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:55:32.379083 10010 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:55:32.379534 10010 webserver.cc:460] Webserver started at http://127.30.148.190:39181/ using document root <none> and password file <none>
I20260619 16:55:32.379769 10010 fs_manager.cc:359] Metadata directory not provided
I20260619 16:55:32.379807 10010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:55:32.379930 10010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 16:55:32 dist-test-slave-xb9n krb5kdc[9566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:55:32.422819 10010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "0b72ab318b90429c852aee534b88239b"
format_stamp: "Formatted at 2026-06-19 16:55:32 on dist-test-slave-xb9n"
server_key: "0de59cbd11fa81d446d45ea5c8622ca7"
server_key_iv: "3950effd9fbcf1b499b9c55e04c5bd5a"
server_key_version: "kuduclusterkey@0"
I20260619 16:55:32.423226 10010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "0b72ab318b90429c852aee534b88239b"
format_stamp: "Formatted at 2026-06-19 16:55:32 on dist-test-slave-xb9n"
server_key: "0de59cbd11fa81d446d45ea5c8622ca7"
server_key_iv: "3950effd9fbcf1b499b9c55e04c5bd5a"
server_key_version: "kuduclusterkey@0"
I20260619 16:55:32.424816 10010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 16:55:32.435595 10025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:55:32.435778 10010 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:55:32.435859 10010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "0b72ab318b90429c852aee534b88239b"
format_stamp: "Formatted at 2026-06-19 16:55:32 on dist-test-slave-xb9n"
server_key: "0de59cbd11fa81d446d45ea5c8622ca7"
server_key_iv: "3950effd9fbcf1b499b9c55e04c5bd5a"
server_key_version: "kuduclusterkey@0"
I20260619 16:55:32.435933 10010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:55:32.452811 10010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:55:32.454831 10010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:55:32.454983 10010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:55:32.459689 10010 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:39031
I20260619 16:55:32.459712 10077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:39031 every 8 connection(s)
I20260619 16:55:32.460031 10010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:55:32.460669 10078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:55:32.462823 10078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b: Bootstrap starting.
I20260619 16:55:32.463353 10078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b: Neither blocks nor log segments found. Creating new log.
I20260619 16:55:32.463644 10078 log.cc:824] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b: Log is configured to *not* fsync() on all Append() calls
I20260619 16:55:32.464268 10078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b: No bootstrap required, opened a new log
I20260619 16:55:32.465415 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 10010
I20260619 16:55:32.465524 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:55:32.465721 31314 external_mini_cluster.cc:1428] Setting key 27cfb6973bd0abfe6cfe748fe248068d
I20260619 16:55:32.466599 10078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b72ab318b90429c852aee534b88239b" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39031 } }
I20260619 16:55:32.466821 10078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:55:32.466900 10078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b72ab318b90429c852aee534b88239b, State: Initialized, Role: FOLLOWER
I20260619 16:55:32.467042 10078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b72ab318b90429c852aee534b88239b" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39031 } }
I20260619 16:55:32.467131 10078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:55:32.467201 10078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:55:32.467258 10078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:55:32.467868 10078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b72ab318b90429c852aee534b88239b" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39031 } }
I20260619 16:55:32.467983 10078 leader_election.cc:302] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b72ab318b90429c852aee534b88239b; no voters:
I20260619 16:55:32.468207 10078 leader_election.cc:288] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:55:32.468247 10083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:55:32.468472 10083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [term 1 LEADER]: Becoming Leader. State: Replica: 0b72ab318b90429c852aee534b88239b, State: Running, Role: LEADER
I20260619 16:55:32.468518 10078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:55:32.469974 10083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b72ab318b90429c852aee534b88239b" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39031 } }
I20260619 16:55:32.470530 10078 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 16:55:32.470527 10084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b72ab318b90429c852aee534b88239b. Latest consensus state: current_term: 1 leader_uuid: "0b72ab318b90429c852aee534b88239b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b72ab318b90429c852aee534b88239b" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39031 } } }
I20260619 16:55:32.470844 10084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [sys.catalog]: This master's current role is: LEADER
I20260619 16:55:32.470932 10085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b72ab318b90429c852aee534b88239b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b72ab318b90429c852aee534b88239b" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39031 } } }
I20260619 16:55:32.471004 10085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b [sys.catalog]: This master's current role is: LEADER
I20260619 16:55:32.473905 10098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 16:55:32 dist-test-slave-xb9n krb5kdc[9566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:55:32.478137 10098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:55:32.480304 10082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:32.466216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44400 (local address 127.30.148.190:39031)
0619 16:55:32.466383 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 16:55:32.466392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:55:32.466450 (+ 58us) server_negotiation.cc:407] Connection header received
0619 16:55:32.466573 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:55:32.466580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:55:32.466632 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:55:32.466770 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:55:32.467566 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:32.470117 (+ 2551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:32.471156 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:32.473674 (+ 2518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:32.474161 (+ 487us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:32.476635 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:55:32.476639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:55:32.476639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:55:32.476651 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:55:32.478646 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:55:32.479044 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:55:32.479047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:55:32.479047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:55:32.479153 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:55:32.479352 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:55:32.479352 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:55:32.479353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:55:32.479637 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:55:32.479772 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 16:55:32.479976 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 16:55:32.480099 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260619 16:55:32.481805 10098 catalog_manager.cc:1347] Generated new cluster ID: 959c350fe53b45889ebb54bef993e447
I20260619 16:55:32.481895 10098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:55:32.498685 10098 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:55:32.499416 10098 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:55:32.504886 10098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b: Generated new TSK 0
I20260619 16:55:32.505086 10098 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:55:33.243911 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.148.148:43721/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39031
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:55:33.349864 10123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:55:33.350039 10123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:55:33.350060 10123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:55:33.351179 10123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:55:33.351219 10123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:55:33.351244 10123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:55:33.352610 10123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.148.148:43721/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:55:33.352824 10123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:55:33.353047 10123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:55:33.357858 10131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:55:33.357856 10129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:55:33.358078 10128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:55:33.358098 10123 server_base.cc:1034] running on GCE node
I20260619 16:55:33.358305 10123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:55:33.358556 10123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:55:33.359711 10123 hybrid_clock.cc:648] HybridClock initialized: now 1781888133359697 us; error 29 us; skew 500 ppm
Jun 19 16:55:33 dist-test-slave-xb9n krb5kdc[9566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:55:33.362229 10123 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:55:33.362633 10123 webserver.cc:460] Webserver started at http://127.30.148.129:38365/ using document root <none> and password file <none>
I20260619 16:55:33.362872 10123 fs_manager.cc:359] Metadata directory not provided
I20260619 16:55:33.362937 10123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:55:33.363066 10123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 16:55:33 dist-test-slave-xb9n krb5kdc[9566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:55:33.376292 10123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "afe01d65df5843f387a4f4b6abb9323c"
format_stamp: "Formatted at 2026-06-19 16:55:33 on dist-test-slave-xb9n"
server_key: "22704a197015e7ae2e229581ca246c80"
server_key_iv: "5d1a6b4946ba7c1ecc370f28e00e0a33"
server_key_version: "kuduclusterkey@0"
I20260619 16:55:33.378602 10123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "afe01d65df5843f387a4f4b6abb9323c"
format_stamp: "Formatted at 2026-06-19 16:55:33 on dist-test-slave-xb9n"
server_key: "22704a197015e7ae2e229581ca246c80"
server_key_iv: "5d1a6b4946ba7c1ecc370f28e00e0a33"
server_key_version: "kuduclusterkey@0"
I20260619 16:55:33.380336 10123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:55:33.393888 10140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:55:33.394089 10123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 16:55:33.394172 10123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "afe01d65df5843f387a4f4b6abb9323c"
format_stamp: "Formatted at 2026-06-19 16:55:33 on dist-test-slave-xb9n"
server_key: "22704a197015e7ae2e229581ca246c80"
server_key_iv: "5d1a6b4946ba7c1ecc370f28e00e0a33"
server_key_version: "kuduclusterkey@0"
I20260619 16:55:33.394246 10123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:55:33.417155 10123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:55:33.419665 10123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:55:33.419864 10123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:55:33.420123 10123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:55:33.420526 10123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:55:33.420646 10123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:55:33.420768 10123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:55:33.420847 10123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:55:33.427335 10123 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:41503
I20260619 16:55:33.427839 10123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:55:33.428990 10253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:41503 every 8 connection(s)
I20260619 16:55:33.429982 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 10123
I20260619 16:55:33.430087 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:55:33.430322 31314 external_mini_cluster.cc:1428] Setting key 085a60335a3fcd840408bfabe00e46aa
Jun 19 16:55:33 dist-test-slave-xb9n krb5kdc[9566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:55:33.443197 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.429598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:39031 (local address 127.30.148.129:45207)
0619 16:55:33.429795 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 16:55:33.430609 (+ 814us) client_negotiation.cc:174] Beginning negotiation
0619 16:55:33.432860 (+ 2251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:55:33.433248 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:55:33.433253 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:55:33.433523 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:55:33.433814 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:33.433821 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.434807 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.434808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:33.436081 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:33.436084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.436644 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.436645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:33.436783 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:33.437406 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:55:33.437415 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:55:33.439202 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:55:33.441479 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:55:33.441486 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:55:33.441488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:55:33.441822 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:55:33.442043 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:55:33.442044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:55:33.442044 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:55:33.442189 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:55:33.442487 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:55:33.442493 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:55:33.442664 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 16:55:33.442774 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 16:55:33.442920 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":87,"threads_started":1}
I20260619 16:55:33.443632 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.429904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:45207 (local address 127.30.148.190:39031)
0619 16:55:33.430037 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 16:55:33.430040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:55:33.432589 (+ 2549us) server_negotiation.cc:407] Connection header received
0619 16:55:33.433035 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:55:33.433037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:55:33.433049 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:55:33.433113 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:55:33.433928 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.434694 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.436181 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.436486 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.436574 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:33.439302 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:55:33.439303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:55:33.439303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:55:33.439310 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:55:33.441333 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:55:33.441910 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:55:33.441912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:55:33.441912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:55:33.441974 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:55:33.442262 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:55:33.442263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:55:33.442263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:55:33.442396 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:55:33.442459 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 16:55:33.443434 (+ 975us) server_negotiation.cc:299] Negotiation successful
0619 16:55:33.443514 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
I20260619 16:55:33.443981 10254 heartbeater.cc:347] Connected to a master server at 127.30.148.190:39031
I20260619 16:55:33.444073 10254 heartbeater.cc:464] Registering TS with master...
I20260619 16:55:33.444243 10254 heartbeater.cc:511] Master 127.30.148.190:39031 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
I20260619 16:55:33.444743 10042 ts_manager.cc:194] Registered new tserver with Master: afe01d65df5843f387a4f4b6abb9323c (127.30.148.129:41503)
I20260619 16:55:33.445624 10042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:45207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:55:33.478094 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.148.148:43721/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39031
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 16:55:33 dist-test-slave-xb9n krb5kdc[9566](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781888133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 16:55:33.596457 10260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:55:33.596791 10260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:55:33.596866 10260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:55:33.598690 10260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:55:33.598794 10260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:55:33.598872 10260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:55:33.601056 10260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.148.148:43721/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:55:33.601562 10260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:55:33.601871 10260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:55:33.608475 10267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:55:33.608916 10270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:55:33.609560 10268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:55:33.612578 10260 server_base.cc:1034] running on GCE node
I20260619 16:55:33.612910 10260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:55:33.613165 10260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:55:33.614303 10260 hybrid_clock.cc:648] HybridClock initialized: now 1781888133614285 us; error 34 us; skew 500 ppm
Jun 19 16:55:33 dist-test-slave-xb9n krb5kdc[9566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:55:33.617468 10260 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:55:33.617925 10260 webserver.cc:460] Webserver started at http://127.30.148.130:44067/ using document root <none> and password file <none>
I20260619 16:55:33.618173 10260 fs_manager.cc:359] Metadata directory not provided
I20260619 16:55:33.618220 10260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:55:33.618337 10260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 16:55:33 dist-test-slave-xb9n krb5kdc[9566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:55:33.634828 10260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "e05ac25e8307429aa215ab5578a1adf5"
format_stamp: "Formatted at 2026-06-19 16:55:33 on dist-test-slave-xb9n"
server_key: "84f740df32b652b3f11d9013606acfa5"
server_key_iv: "8a3aed09c02b574e518258492acac625"
server_key_version: "kuduclusterkey@0"
I20260619 16:55:33.635344 10260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "e05ac25e8307429aa215ab5578a1adf5"
format_stamp: "Formatted at 2026-06-19 16:55:33 on dist-test-slave-xb9n"
server_key: "84f740df32b652b3f11d9013606acfa5"
server_key_iv: "8a3aed09c02b574e518258492acac625"
server_key_version: "kuduclusterkey@0"
I20260619 16:55:33.638904 10260 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260619 16:55:33.654613 10277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:55:33.654842 10260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:55:33.654944 10260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "e05ac25e8307429aa215ab5578a1adf5"
format_stamp: "Formatted at 2026-06-19 16:55:33 on dist-test-slave-xb9n"
server_key: "84f740df32b652b3f11d9013606acfa5"
server_key_iv: "8a3aed09c02b574e518258492acac625"
server_key_version: "kuduclusterkey@0"
I20260619 16:55:33.655033 10260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:55:33.667377 10260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:55:33.669863 10260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:55:33.670022 10260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:55:33.670260 10260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:55:33.670667 10260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:55:33.670733 10260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:55:33.670778 10260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:55:33.670809 10260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:55:33.676832 10260 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:38259
I20260619 16:55:33.677268 10260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:55:33.683714 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 10260
I20260619 16:55:33.683827 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:55:33.684020 31314 external_mini_cluster.cc:1428] Setting key aedd6af5189c7899db37ba394a40e58f
I20260619 16:55:33.689771 10391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:38259 every 8 connection(s)
Jun 19 16:55:33 dist-test-slave-xb9n krb5kdc[9566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
I20260619 16:55:33.705713 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.690635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:59253 (local address 127.30.148.190:39031)
0619 16:55:33.690683 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:55:33.690685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:55:33.691348 (+ 663us) server_negotiation.cc:407] Connection header received
0619 16:55:33.692012 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:55:33.692013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:55:33.692028 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:55:33.692101 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:55:33.692844 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.693674 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.695110 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.695411 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.695451 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:33.699221 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:55:33.699223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:55:33.699223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:55:33.699230 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:55:33.701024 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:55:33.704618 (+ 3594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:55:33.704619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:55:33.704620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:55:33.704680 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:55:33.704989 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:55:33.704989 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:55:33.704990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:55:33.705119 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:55:33.705157 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 16:55:33.705531 (+ 374us) server_negotiation.cc:299] Negotiation successful
0619 16:55:33.705619 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
I20260619 16:55:33.706389 10395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.690440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:39031 (local address 127.30.148.130:59253)
0619 16:55:33.691214 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0619 16:55:33.691263 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 16:55:33.691887 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:55:33.692233 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:55:33.692237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:55:33.692471 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:55:33.692736 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:33.692743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.693788 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.693789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:33.694997 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:33.695000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.695537 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.695537 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:33.695674 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:33.697454 (+ 1780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:55:33.697464 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:55:33.699114 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:55:33.701145 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:55:33.701163 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:55:33.701165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:55:33.701483 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:55:33.704754 (+ 3271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:55:33.704754 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:55:33.704755 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:55:33.704902 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:55:33.705197 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:55:33.705202 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:55:33.705367 (+ 165us) client_negotiation.cc:769] Sending connection context
0619 16:55:33.705995 (+ 628us) client_negotiation.cc:240] Negotiation successful
0619 16:55:33.706156 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":414,"threads_started":1}
I20260619 16:55:33.706992 10392 heartbeater.cc:347] Connected to a master server at 127.30.148.190:39031
I20260619 16:55:33.707087 10392 heartbeater.cc:464] Registering TS with master...
I20260619 16:55:33.707252 10392 heartbeater.cc:511] Master 127.30.148.190:39031 requested a full tablet report, sending...
I20260619 16:55:33.707594 10042 ts_manager.cc:194] Registered new tserver with Master: e05ac25e8307429aa215ab5578a1adf5 (127.30.148.130:38259)
I20260619 16:55:33.708041 10042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:59253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:55:33.730157 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.148.148:43721/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39031
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:55:33.847895 10397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:55:33.848227 10397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:55:33.848275 10397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:55:33.849903 10397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:55:33.850004 10397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:55:33.850090 10397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:55:33.852142 10397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.148.148:43721/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:55:33.852556 10397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:55:33.852825 10397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:55:33.859015 10403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:55:33.859009 10405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:55:33.859560 10402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:55:33.860620 10397 server_base.cc:1034] running on GCE node
I20260619 16:55:33.860828 10397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:55:33.861085 10397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:55:33.865541 10397 hybrid_clock.cc:648] HybridClock initialized: now 1781888133865509 us; error 46 us; skew 500 ppm
Jun 19 16:55:33 dist-test-slave-xb9n krb5kdc[9566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:55:33.868780 10397 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:55:33.869397 10397 webserver.cc:460] Webserver started at http://127.30.148.131:34687/ using document root <none> and password file <none>
I20260619 16:55:33.869720 10397 fs_manager.cc:359] Metadata directory not provided
I20260619 16:55:33.869820 10397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:55:33.870000 10397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 16:55:33 dist-test-slave-xb9n krb5kdc[9566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:55:33.888978 10397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "c1c2c26d74c44010a7834070e8974a61"
format_stamp: "Formatted at 2026-06-19 16:55:33 on dist-test-slave-xb9n"
server_key: "f8e28e1a3911db6b080ee05fd0497a9c"
server_key_iv: "04f554a884b29aa1c08c8602bf18d66c"
server_key_version: "kuduclusterkey@0"
I20260619 16:55:33.889454 10397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "c1c2c26d74c44010a7834070e8974a61"
format_stamp: "Formatted at 2026-06-19 16:55:33 on dist-test-slave-xb9n"
server_key: "f8e28e1a3911db6b080ee05fd0497a9c"
server_key_iv: "04f554a884b29aa1c08c8602bf18d66c"
server_key_version: "kuduclusterkey@0"
I20260619 16:55:33.892201 10397 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260619 16:55:33.905118 10412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:55:33.905342 10397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 16:55:33.905509 10397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "c1c2c26d74c44010a7834070e8974a61"
format_stamp: "Formatted at 2026-06-19 16:55:33 on dist-test-slave-xb9n"
server_key: "f8e28e1a3911db6b080ee05fd0497a9c"
server_key_iv: "04f554a884b29aa1c08c8602bf18d66c"
server_key_version: "kuduclusterkey@0"
I20260619 16:55:33.905592 10397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:55:33.923919 10397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:55:33.929946 10397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:55:33.930164 10397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:55:33.930429 10397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:55:33.936040 10397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:55:33.936124 10397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:55:33.936189 10397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:55:33.936226 10397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:55:33.946787 10397 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:37343
I20260619 16:55:33.947266 10397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:55:33.950922 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 10397
I20260619 16:55:33.951023 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:55:33.951251 31314 external_mini_cluster.cc:1428] Setting key d2c8a430133bf1412224ca75fa6350b6
I20260619 16:55:33.956794 10525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:37343 every 8 connection(s)
Jun 19 16:55:33 dist-test-slave-xb9n krb5kdc[9566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:55:33.968475 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.957794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:32999 (local address 127.30.148.190:39031)
0619 16:55:33.957842 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:55:33.957844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:55:33.957919 (+ 75us) server_negotiation.cc:407] Connection header received
0619 16:55:33.958587 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:55:33.958588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:55:33.958599 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:55:33.958671 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:55:33.959391 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.960137 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.961607 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.961882 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.961929 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:33.964883 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:55:33.964884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:55:33.964884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:55:33.964891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:55:33.966726 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:55:33.967234 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:55:33.967235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:55:33.967235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:55:33.967316 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:55:33.967733 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:55:33.967734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:55:33.967734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:55:33.967869 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:55:33.967947 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 16:55:33.968301 (+ 354us) server_negotiation.cc:299] Negotiation successful
0619 16:55:33.968382 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 16:55:33.968878 10528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.957495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:39031 (local address 127.30.148.131:32999)
0619 16:55:33.957804 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0619 16:55:33.957847 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 16:55:33.958468 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:55:33.958756 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:55:33.958761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:55:33.958988 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:55:33.959278 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:33.959285 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.960264 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.960265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:33.961507 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:33.961510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.962019 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.962020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:33.962168 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:33.962980 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:55:33.963021 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:55:33.964758 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:55:33.966834 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:55:33.966841 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:55:33.966843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:55:33.967134 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:55:33.967396 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:55:33.967397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:55:33.967397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:55:33.967548 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:55:33.967958 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:55:33.967965 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:55:33.968121 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 16:55:33.968252 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 16:55:33.968398 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":92,"threads_started":1}
I20260619 16:55:33.969655 10526 heartbeater.cc:347] Connected to a master server at 127.30.148.190:39031
I20260619 16:55:33.969794 10526 heartbeater.cc:464] Registering TS with master...
I20260619 16:55:33.970003 10526 heartbeater.cc:511] Master 127.30.148.190:39031 requested a full tablet report, sending...
I20260619 16:55:33.970439 10042 ts_manager.cc:194] Registered new tserver with Master: c1c2c26d74c44010a7834070e8974a61 (127.30.148.131:37343)
I20260619 16:55:33.970919 10042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:32999
I20260619 16:55:33.973766 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:55:33.985131 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.975403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44412 (local address 127.30.148.190:39031)
0619 16:55:33.975443 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:55:33.975445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:55:33.977520 (+ 2075us) server_negotiation.cc:407] Connection header received
0619 16:55:33.977611 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:55:33.977612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:55:33.977623 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:55:33.977690 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:55:33.978425 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.979175 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.980253 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.980548 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.980626 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:33.982105 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:55:33.982106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:55:33.982106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:55:33.982114 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:55:33.983898 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:55:33.984311 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:55:33.984312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:55:33.984312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:55:33.984372 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:55:33.984603 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:55:33.984604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:55:33.984604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:55:33.984727 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:55:33.984825 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 16:55:33.984965 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 16:55:33.985039 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:55:33.987135 10042 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:55:33.989382 10042 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:55:33.997625 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.992015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:37343 (local address 127.0.0.1:50600)
0619 16:55:33.993002 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0619 16:55:33.993040 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:55:33.993097 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:55:33.993733 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:55:33.993736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:55:33.993738 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:55:33.993795 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:33.993796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.995039 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.995040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:33.996317 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:33.996319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.997329 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.997330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:33.997433 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:33.997445 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 16:55:33.997509 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 16:55:33.997554 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"thread_start_us":64,"threads_started":1}
I20260619 16:55:33.998100 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.992576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50600 (local address 127.30.148.131:37343)
0619 16:55:33.993488 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0619 16:55:33.993492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:55:33.993503 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:55:33.993537 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:55:33.993539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:55:33.993555 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:55:33.993635 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:55:33.993867 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.994946 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.996434 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:33.997163 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:33.997245 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:33.997296 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:55:33.997971 (+ 675us) server_negotiation.cc:299] Negotiation successful
0619 16:55:33.998010 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":83,"threads_started":1}
I20260619 16:55:33.998767 10446 tablet_service.cc:1467] Processing CreateTablet for tablet dcbd2c1e6fc744d3ad440751039209f1 (DEFAULT_TABLE table=test-table [id=3e8ac2007eb64cf881345a9afa05fb22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:55:33.999097 10446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbd2c1e6fc744d3ad440751039209f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:55:34.002157 10544 tablet_bootstrap.cc:492] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61: Bootstrap starting.
I20260619 16:55:34.003275 10544 tablet_bootstrap.cc:654] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61: Neither blocks nor log segments found. Creating new log.
I20260619 16:55:34.004010 10544 log.cc:824] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61: Log is configured to *not* fsync() on all Append() calls
I20260619 16:55:34.006126 10544 tablet_bootstrap.cc:492] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61: No bootstrap required, opened a new log
I20260619 16:55:34.006199 10544 ts_tablet_manager.cc:1397] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 16:55:34.008105 10544 raft_consensus.cc:348] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c2c26d74c44010a7834070e8974a61" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37343 } } peers { permanent_uuid: "e05ac25e8307429aa215ab5578a1adf5" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38259 } } peers { permanent_uuid: "afe01d65df5843f387a4f4b6abb9323c" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 41503 } }
I20260619 16:55:34.008282 10544 raft_consensus.cc:374] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:55:34.008323 10544 raft_consensus.cc:729] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1c2c26d74c44010a7834070e8974a61, State: Initialized, Role: FOLLOWER
I20260619 16:55:34.008430 10544 consensus_queue.cc:260] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c2c26d74c44010a7834070e8974a61" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37343 } } peers { permanent_uuid: "e05ac25e8307429aa215ab5578a1adf5" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38259 } } peers { permanent_uuid: "afe01d65df5843f387a4f4b6abb9323c" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 41503 } }
I20260619 16:55:34.008668 10544 ts_tablet_manager.cc:1428] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:55:34.008889 10526 heartbeater.cc:503] Master 127.30.148.190:39031 was elected leader, sending a full tablet report...
I20260619 16:55:34.009159 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.992372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:41503 (local address 127.0.0.1:59666)
0619 16:55:33.993317 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0619 16:55:33.993324 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 16:55:33.993382 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:55:34.005460 (+ 12078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:55:34.005461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:55:34.005464 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:55:34.005506 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:34.005507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.006866 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.006867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:34.008010 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:34.008012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.008940 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.008941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:34.009003 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:34.009007 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:55:34.009035 (+ 28us) client_negotiation.cc:240] Negotiation successful
0619 16:55:34.009084 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"thread_start_us":63,"threads_started":1}
I20260619 16:55:34.009505 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.992481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59666 (local address 127.30.148.129:41503)
0619 16:55:34.000313 (+ 7832us) server_negotiation.cc:206] Beginning negotiation
0619 16:55:34.000318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:55:34.000329 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:55:34.000360 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:55:34.000363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:55:34.000379 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:55:34.000457 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:55:34.005673 (+ 5216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.006765 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.008122 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.008845 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.009287 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:34.009350 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 16:55:34.009396 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 16:55:34.009433 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7799,"thread_start_us":75,"threads_started":1}
I20260619 16:55:34.010505 10176 tablet_service.cc:1467] Processing CreateTablet for tablet dcbd2c1e6fc744d3ad440751039209f1 (DEFAULT_TABLE table=test-table [id=3e8ac2007eb64cf881345a9afa05fb22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:55:34.010630 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.992212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:38259 (local address 127.0.0.1:34626)
0619 16:55:33.993208 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0619 16:55:33.993215 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 16:55:33.993251 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:55:34.006905 (+ 13654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:55:34.006906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:55:34.006908 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:55:34.006948 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:34.006950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.008270 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.008271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:34.009351 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:34.009353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.010412 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.010414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:34.010478 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:34.010482 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:55:34.010512 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 16:55:34.010551 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"thread_start_us":58,"threads_started":1}
I20260619 16:55:34.010805 10176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbd2c1e6fc744d3ad440751039209f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:55:34.011087 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:33.992678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.30.148.130:38259)
0619 16:55:34.005492 (+ 12814us) server_negotiation.cc:206] Beginning negotiation
0619 16:55:34.005497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:55:34.005506 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:55:34.005543 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:55:34.005545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:55:34.005562 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:55:34.007026 (+ 1464us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:55:34.007105 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.008168 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.009487 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.010316 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.010861 (+ 545us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:34.010931 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 16:55:34.010959 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 16:55:34.010993 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12771,"thread_start_us":79,"threads_started":1}
I20260619 16:55:34.011663 10320 tablet_service.cc:1467] Processing CreateTablet for tablet dcbd2c1e6fc744d3ad440751039209f1 (DEFAULT_TABLE table=test-table [id=3e8ac2007eb64cf881345a9afa05fb22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:55:34.011966 10320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcbd2c1e6fc744d3ad440751039209f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:55:34.013959 10547 tablet_bootstrap.cc:492] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c: Bootstrap starting.
I20260619 16:55:34.014587 10547 tablet_bootstrap.cc:654] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c: Neither blocks nor log segments found. Creating new log.
I20260619 16:55:34.014855 10547 log.cc:824] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c: Log is configured to *not* fsync() on all Append() calls
I20260619 16:55:34.015512 10547 tablet_bootstrap.cc:492] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c: No bootstrap required, opened a new log
I20260619 16:55:34.015573 10547 ts_tablet_manager.cc:1397] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:55:34.017851 10547 raft_consensus.cc:348] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c2c26d74c44010a7834070e8974a61" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37343 } } peers { permanent_uuid: "e05ac25e8307429aa215ab5578a1adf5" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38259 } } peers { permanent_uuid: "afe01d65df5843f387a4f4b6abb9323c" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 41503 } }
I20260619 16:55:34.018003 10547 raft_consensus.cc:374] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:55:34.018031 10547 raft_consensus.cc:729] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afe01d65df5843f387a4f4b6abb9323c, State: Initialized, Role: FOLLOWER
I20260619 16:55:34.018142 10547 consensus_queue.cc:260] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c2c26d74c44010a7834070e8974a61" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37343 } } peers { permanent_uuid: "e05ac25e8307429aa215ab5578a1adf5" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38259 } } peers { permanent_uuid: "afe01d65df5843f387a4f4b6abb9323c" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 41503 } }
I20260619 16:55:34.018354 10547 ts_tablet_manager.cc:1428] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:55:34.018461 10254 heartbeater.cc:503] Master 127.30.148.190:39031 was elected leader, sending a full tablet report...
I20260619 16:55:34.019412 10549 tablet_bootstrap.cc:492] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5: Bootstrap starting.
I20260619 16:55:34.020278 10549 tablet_bootstrap.cc:654] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5: Neither blocks nor log segments found. Creating new log.
I20260619 16:55:34.020565 10549 log.cc:824] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5: Log is configured to *not* fsync() on all Append() calls
I20260619 16:55:34.021338 10549 tablet_bootstrap.cc:492] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5: No bootstrap required, opened a new log
I20260619 16:55:34.021425 10549 ts_tablet_manager.cc:1397] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:55:34.023336 10549 raft_consensus.cc:348] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c2c26d74c44010a7834070e8974a61" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37343 } } peers { permanent_uuid: "e05ac25e8307429aa215ab5578a1adf5" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38259 } } peers { permanent_uuid: "afe01d65df5843f387a4f4b6abb9323c" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 41503 } }
I20260619 16:55:34.023497 10549 raft_consensus.cc:374] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:55:34.023526 10549 raft_consensus.cc:729] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e05ac25e8307429aa215ab5578a1adf5, State: Initialized, Role: FOLLOWER
I20260619 16:55:34.023672 10549 consensus_queue.cc:260] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c2c26d74c44010a7834070e8974a61" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37343 } } peers { permanent_uuid: "e05ac25e8307429aa215ab5578a1adf5" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38259 } } peers { permanent_uuid: "afe01d65df5843f387a4f4b6abb9323c" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 41503 } }
I20260619 16:55:34.023896 10549 ts_tablet_manager.cc:1428] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 16:55:34.023993 10392 heartbeater.cc:503] Master 127.30.148.190:39031 was elected leader, sending a full tablet report...
I20260619 16:55:34.033553 10550 raft_consensus.cc:482] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:55:34.033666 10550 raft_consensus.cc:504] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c2c26d74c44010a7834070e8974a61" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37343 } } peers { permanent_uuid: "e05ac25e8307429aa215ab5578a1adf5" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38259 } } peers { permanent_uuid: "afe01d65df5843f387a4f4b6abb9323c" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 41503 } }
I20260619 16:55:34.033970 10550 leader_election.cc:288] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1c2c26d74c44010a7834070e8974a61 (127.30.148.131:37343), e05ac25e8307429aa215ab5578a1adf5 (127.30.148.130:38259)
I20260619 16:55:34.037953 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:34.034502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:59331 (local address 127.30.148.131:37343)
0619 16:55:34.034541 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:55:34.034543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:55:34.034552 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:55:34.034673 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:55:34.034674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:55:34.034683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:55:34.034713 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:55:34.034942 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.035664 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.037021 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.037584 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.037632 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:34.037640 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:55:34.037848 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 16:55:34.037887 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:55:34.038353 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:34.034362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:60685 (local address 127.30.148.130:38259)
0619 16:55:34.034401 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:55:34.034403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:55:34.034569 (+ 166us) server_negotiation.cc:407] Connection header received
0619 16:55:34.034664 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:55:34.034665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:55:34.034674 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:55:34.034702 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:55:34.034906 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.035648 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.037459 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.037987 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.038029 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:34.038036 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:55:34.038258 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 16:55:34.038292 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:55:34.038360 10553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:34.034134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:37343 (local address 127.30.148.129:59331)
0619 16:55:34.034420 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 16:55:34.034430 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:55:34.034587 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:55:34.034778 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:55:34.034779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:55:34.034781 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:55:34.034864 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:34.034866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.035786 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.035786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:34.036915 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:34.036918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.037698 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.037699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:34.037768 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:34.037771 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:55:34.038225 (+ 454us) client_negotiation.cc:240] Negotiation successful
0619 16:55:34.038270 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260619 16:55:34.038543 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:34.034308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:38259 (local address 127.30.148.129:60685)
0619 16:55:34.034511 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 16:55:34.034521 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:55:34.034596 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:55:34.034760 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:55:34.034760 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:55:34.034762 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:55:34.034840 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:34.034842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.036210 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.036211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:34.037339 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:55:34.037341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.038102 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.038103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:55:34.038183 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:34.038186 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:55:34.038427 (+ 241us) client_negotiation.cc:240] Negotiation successful
0619 16:55:34.038468 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":2,"thread_start_us":54,"threads_started":1}
I20260619 16:55:34.039077 10346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbd2c1e6fc744d3ad440751039209f1" candidate_uuid: "afe01d65df5843f387a4f4b6abb9323c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e05ac25e8307429aa215ab5578a1adf5" is_pre_election: true
I20260619 16:55:34.039074 10480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbd2c1e6fc744d3ad440751039209f1" candidate_uuid: "afe01d65df5843f387a4f4b6abb9323c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1c2c26d74c44010a7834070e8974a61" is_pre_election: true
I20260619 16:55:34.039206 10346 raft_consensus.cc:2454] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afe01d65df5843f387a4f4b6abb9323c in term 0.
I20260619 16:55:34.039209 10480 raft_consensus.cc:2454] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afe01d65df5843f387a4f4b6abb9323c in term 0.
I20260619 16:55:34.039530 10142 leader_election.cc:302] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afe01d65df5843f387a4f4b6abb9323c, c1c2c26d74c44010a7834070e8974a61; no voters:
I20260619 16:55:34.039690 10550 raft_consensus.cc:2781] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:55:34.039727 10550 raft_consensus.cc:482] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:55:34.039748 10550 raft_consensus.cc:3037] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:55:34.040602 10550 raft_consensus.cc:504] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c2c26d74c44010a7834070e8974a61" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37343 } } peers { permanent_uuid: "e05ac25e8307429aa215ab5578a1adf5" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38259 } } peers { permanent_uuid: "afe01d65df5843f387a4f4b6abb9323c" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 41503 } }
I20260619 16:55:34.040709 10550 leader_election.cc:288] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [CANDIDATE]: Term 1 election: Requested vote from peers c1c2c26d74c44010a7834070e8974a61 (127.30.148.131:37343), e05ac25e8307429aa215ab5578a1adf5 (127.30.148.130:38259)
I20260619 16:55:34.041456 10480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbd2c1e6fc744d3ad440751039209f1" candidate_uuid: "afe01d65df5843f387a4f4b6abb9323c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1c2c26d74c44010a7834070e8974a61"
I20260619 16:55:34.041530 10480 raft_consensus.cc:3037] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:55:34.042325 10480 raft_consensus.cc:2454] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afe01d65df5843f387a4f4b6abb9323c in term 1.
I20260619 16:55:34.042508 10346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcbd2c1e6fc744d3ad440751039209f1" candidate_uuid: "afe01d65df5843f387a4f4b6abb9323c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e05ac25e8307429aa215ab5578a1adf5"
I20260619 16:55:34.042572 10346 raft_consensus.cc:3037] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:55:34.043362 10346 raft_consensus.cc:2454] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afe01d65df5843f387a4f4b6abb9323c in term 1.
I20260619 16:55:34.043514 10142 leader_election.cc:302] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afe01d65df5843f387a4f4b6abb9323c, c1c2c26d74c44010a7834070e8974a61; no voters:
I20260619 16:55:34.043630 10550 raft_consensus.cc:2781] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:55:34.043807 10550 raft_consensus.cc:686] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [term 1 LEADER]: Becoming Leader. State: Replica: afe01d65df5843f387a4f4b6abb9323c, State: Running, Role: LEADER
I20260619 16:55:34.043891 10550 consensus_queue.cc:237] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c2c26d74c44010a7834070e8974a61" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37343 } } peers { permanent_uuid: "e05ac25e8307429aa215ab5578a1adf5" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38259 } } peers { permanent_uuid: "afe01d65df5843f387a4f4b6abb9323c" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 41503 } }
I20260619 16:55:34.044616 10042 catalog_manager.cc:5495] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c reported cstate change: term changed from 0 to 1, leader changed from <none> to afe01d65df5843f387a4f4b6abb9323c (127.30.148.129). New cstate: current_term: 1 leader_uuid: "afe01d65df5843f387a4f4b6abb9323c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c2c26d74c44010a7834070e8974a61" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e05ac25e8307429aa215ab5578a1adf5" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afe01d65df5843f387a4f4b6abb9323c" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 41503 } health_report { overall_health: HEALTHY } } }
I20260619 16:55:34.057822 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:34.054710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59678 (local address 127.30.148.129:41503)
0619 16:55:34.054751 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 16:55:34.054753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:55:34.054762 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:55:34.054819 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:55:34.054819 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:55:34.054837 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:55:34.054913 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:55:34.055052 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.055762 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.056829 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:55:34.057163 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:55:34.057220 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:55:34.057424 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:55:34.057541 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:55:34.057622 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 16:55:34.057714 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 16:55:34.057736 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:55:34.059588 10346 raft_consensus.cc:1261] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5 [term 1 FOLLOWER]: Refusing update from remote peer afe01d65df5843f387a4f4b6abb9323c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:55:34.059605 10480 raft_consensus.cc:1261] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61 [term 1 FOLLOWER]: Refusing update from remote peer afe01d65df5843f387a4f4b6abb9323c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:55:34.059957 10550 consensus_queue.cc:1035] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e05ac25e8307429aa215ab5578a1adf5" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 38259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:55:34.060066 10555 consensus_queue.cc:1035] T dcbd2c1e6fc744d3ad440751039209f1 P afe01d65df5843f387a4f4b6abb9323c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1c2c26d74c44010a7834070e8974a61" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 37343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:55:34.068827 10558 mvcc.cc:205] Tried to move back new op lower bound from 7298613797105700864 to 7298613797044305920. Current Snapshot: MvccSnapshot[applied={T|T < 7298613797105700864}]
I20260619 16:55:34.068969 10560 mvcc.cc:205] Tried to move back new op lower bound from 7298613797105700864 to 7298613797044305920. Current Snapshot: MvccSnapshot[applied={T|T < 7298613797105700864}]
I20260619 16:55:34.070696 10559 mvcc.cc:205] Tried to move back new op lower bound from 7298613797105700864 to 7298613797044305920. Current Snapshot: MvccSnapshot[applied={T|T < 7298613797105700864}]
I20260619 16:55:34.074951 10041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:55:34.075064 10041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:55:34.075852 10041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b72ab318b90429c852aee534b88239b: Sending DeleteTablet for 3 replicas of tablet dcbd2c1e6fc744d3ad440751039209f1
I20260619 16:55:34.076344 10446 tablet_service.cc:1514] Processing DeleteTablet for tablet dcbd2c1e6fc744d3ad440751039209f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:55:34 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:50600
I20260619 16:55:34.076553 10320 tablet_service.cc:1514] Processing DeleteTablet for tablet dcbd2c1e6fc744d3ad440751039209f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:55:34 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:34626
I20260619 16:55:34.076758 10176 tablet_service.cc:1514] Processing DeleteTablet for tablet dcbd2c1e6fc744d3ad440751039209f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:55:34 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:59666
I20260619 16:55:34.077303 10567 tablet_replica.cc:330] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61: stopping tablet replica
I20260619 16:55:34.077394 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 10123
I20260619 16:55:34.077447 10567 raft_consensus.cc:2229] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:55:34.077629 10567 raft_consensus.cc:2258] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:55:34.078998 10567 ts_tablet_manager.cc:1905] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:55:34.080647 10567 ts_tablet_manager.cc:1918] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:55:34.080766 10567 log.cc:1196] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/dcbd2c1e6fc744d3ad440751039209f1
I20260619 16:55:34.080928 10568 tablet_replica.cc:330] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5: stopping tablet replica
I20260619 16:55:34.081032 10568 raft_consensus.cc:2229] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:55:34.081138 10568 raft_consensus.cc:2258] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:55:34.081633 10568 ts_tablet_manager.cc:1905] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:55:34.081943 10567 ts_tablet_manager.cc:1939] T dcbd2c1e6fc744d3ad440751039209f1 P c1c2c26d74c44010a7834070e8974a61: Deleting consensus metadata
I20260619 16:55:34.083475 10568 ts_tablet_manager.cc:1918] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:55:34.084069 10027 catalog_manager.cc:4841] TS c1c2c26d74c44010a7834070e8974a61 (127.30.148.131:37343): tablet dcbd2c1e6fc744d3ad440751039209f1 (table test-table [id=3e8ac2007eb64cf881345a9afa05fb22]) successfully deleted
I20260619 16:55:34.084215 10568 log.cc:1196] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/dcbd2c1e6fc744d3ad440751039209f1
I20260619 16:55:34.084578 10568 ts_tablet_manager.cc:1939] T dcbd2c1e6fc744d3ad440751039209f1 P e05ac25e8307429aa215ab5578a1adf5: Deleting consensus metadata
I20260619 16:55:34.085120 10027 catalog_manager.cc:4841] TS e05ac25e8307429aa215ab5578a1adf5 (127.30.148.130:38259): tablet dcbd2c1e6fc744d3ad440751039209f1 (table test-table [id=3e8ac2007eb64cf881345a9afa05fb22]) successfully deleted
Jun 19 16:55:34 dist-test-slave-xb9n krb5kdc[9566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
W20260619 16:55:34.085692 10028 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:41503 (error 108)
I20260619 16:55:34.086131 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:55:34.086034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:41503 (local address 127.0.0.1:59680)
0619 16:55:34.086071 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 16:55:34.086102 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:41503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260619 16:55:34.086259 10028 catalog_manager.cc:4584] TS afe01d65df5843f387a4f4b6abb9323c (127.30.148.129:41503): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dcbd2c1e6fc744d3ad440751039209f1: Network error: Client connection negotiation failed: client connection to 127.30.148.129:41503: connect: Connection refused (error 111)
I20260619 16:55:34.086570 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 10260
I20260619 16:55:34.091765 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 10397
I20260619 16:55:34.097610 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 10010
I20260619 16:55:34.104346 31314 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 16:55:34.448725 31314 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 16:55:34.448854 31314 mini_ranger.cc:66] Stopping Ranger...
I20260619 16:55:34.793342 31314 mini_ranger.cc:68] Stopped Ranger
2026-06-19 16:55:34.793 UTC [9596] LOG: received smart shutdown request
2026-06-19 16:55:34.797 UTC [9596] LOG: background worker "logical replication launcher" (PID 9605) exited with exit code 1
2026-06-19 16:55:34.797 UTC [9600] LOG: shutting down
2026-06-19 16:55:34.966 UTC [9596] LOG: database system is shut down
2026-06-19T16:55:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (42615 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:55:35 dist-test-slave-xb9n krb5kdc[10579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:55:35 dist-test-slave-xb9n krb5kdc[10579](info): set up 2 sockets
Jun 19 16:55:35 dist-test-slave-xb9n krb5kdc[10579](info): commencing operation
krb5kdc: starting...
W20260619 16:55:36.089632 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:55:36 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:55:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:55:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.148.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.148.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/rangeradmin_127.30.148.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/rangeradmin_127.30.148.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.148.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.148.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/rangerlookup_127.30.148.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/rangerlookup_127.30.148.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.148.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.148.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/HTTP_127.30.148.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/HTTP_127.30.148.148@KRBTEST.COM.keytab.
I20260619 16:55:36.238310 31314 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLxMmiv/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/postgres -l logfile start
2026-06-19 16:55:38.506 UTC [10609] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 16:55:38.506 UTC [10609] LOG: listening on IPv4 address "127.30.148.148", port 44153
2026-06-19 16:55:38.509 UTC [10609] LOG: listening on Unix socket "/tmp/.s.PGSQL.44153"
2026-06-19 16:55:38.529 UTC [10612] LOG: database system was shut down at 2026-06-19 16:55:37 UTC
2026-06-19 16:55:38.532 UTC [10609] LOG: database system is ready to accept connections
I20260619 16:55:39.477106 31314 mini_postgres.cc:96] Postgres bound to 44153
2026-06-19 16:55:39.483 UTC [10620] FATAL: database "slave" does not exist
127.30.148.148:44153 - accepting connections
I20260619 16:55:39.483826 31314 mini_ranger.cc:161] Starting Ranger...
I20260619 16:55:39.497974 31314 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 16:55:39.904093 31314 mini_ranger.cc:87] Created ranger Postgres database
I20260619 16:55:39.904178 31314 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-admin
2026-06-19 16:55:40,171 [I] DB FLAVOR :POSTGRES
2026-06-19 16:55:40,171 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 16:55:40,171 [I] Checking connection..
2026-06-19 16:55:40,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 16:55:40,406 [I] Checking connection passed.
2026-06-19 16:55:40,406 [I] --------- Verifying version history table ---------
2026-06-19 16:55:40,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 16:55:40,661 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 16:55:40,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 16:55:40,890 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 16:55:40,890 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 16:55:40,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 16:55:41.115 UTC [10684] WARNING: there is no transaction in progress
2026-06-19 16:55:41,121 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 16:55:41,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 16:55:41,373 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 16:55:41,374 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 16:55:41,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 16:55:41,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 16:55:41,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-xb9n.c.gcp-upstream.internal','N') ;"
2026-06-19 16:55:42,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 16:55:42,286 [I] Table x_portal_user does not exist in database ranger
2026-06-19 16:55:42,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 16:55:42,531 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 16:55:42,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 16:55:42,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 16:55:42,997 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 16:55:42,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 16:55:43.871 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:43.885 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:43.900 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:43.914 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:43.931 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:43.946 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:43.993 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:43.999 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:44.011 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:44.020 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:44.029 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:44.041 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:44.050 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:44.056 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:44.061 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:44.067 UTC [10792] WARNING: there is no transaction in progress
2026-06-19 16:55:44,606 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 16:55:44,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 16:55:44,820 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 16:55:44,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 16:55:45,069 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 16:55:45,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 16:55:45,323 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 16:55:45,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 16:55:45,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 16:55:45,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-xb9n.c.gcp-upstream.internal';"
2026-06-19 16:55:46,045 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 16:55:46,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 16:55:46,275 [I] DB_PATCHES have already been applied
I20260619 16:55:46.281257 31314 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLxMmiv/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 16:55:46.281317 31314 mini_ranger.cc:193] Using host: 127.30.148.148
I20260619 16:55:46.284610 31314 mini_ranger.cc:227] Ranger admin URL: http://127.30.148.148:43659
Jun 19, 2026 4:55:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 4:55:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 4:55:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 4:55:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 4:55:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 4:55:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 4:55:47 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 16:55:47.375560 31314 mini_ranger.cc:160] Time spent starting Ranger: real 7.892s user 0.000s sys 0.008s
Jun 19, 2026 4:56:05 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 4:56:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Jun 19, 2026 4:56:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 4:56:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 4:56:05 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 4:56:06 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 16:56:07.204095 31314 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.148.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.148.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/rangerkms_127.30.148.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.148.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/rangerkms_127.30.148.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.148.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.148.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.148.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/HTTP_127.30.148.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.148.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/HTTP_127.30.148.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 16:56:07 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888167, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 16:56:07.294480 31314 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 16:56:07.302839 31314 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 16:56:08.059572 31314 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 16:56:08.059652 31314 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-kms
I20260619 16:56:08.059669 31314 mini_ranger_kms.cc:227] Using postgres at 127.30.148.148:44153
2026-06-19 16:56:08,220 [I] DB FLAVOR :POSTGRES
2026-06-19 16:56:08,221 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 16:56:08,221 [I] Checking connection
2026-06-19 16:56:08,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 16:56:08,441 [I] connection success
2026-06-19 16:56:08,441 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 16:56:08,441 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 16:56:08,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 16:56:08,667 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 16:56:08,667 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 16:56:08,667 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 16:56:08,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.148.148:44153/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 16:56:08,904 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 16:56:09.375814 31314 mini_ranger_kms.cc:310] Created kms service
I20260619 16:56:09.560772 31314 mini_ranger_kms.cc:326] Created kudu user
I20260619 16:56:09.603137 31314 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 16:56:09 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888167, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:56:09.974725 31314 mini_ranger_kms.cc:384] Added ranger policy
I20260619 16:56:09.974946 31314 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLxMmiv/build/release/bin/postgresql.jar:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLxMmiv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 16:56:09.974982 31314 mini_ranger_kms.cc:240] Using host: 127.30.148.148
I20260619 16:56:09.978307 31314 mini_ranger_kms.cc:276] Ranger KMS PID: 10972
I20260619 16:56:09.978389 31314 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.148.148:42127
Jun 19, 2026 4:56:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 4:56:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 4:56:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 4:56:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 4:56:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 4:56:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 16:56:12.205312 31314 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.911s user 0.005s sys 0.139s
I20260619 16:56:12.205446 31314 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 16:56:12.205497 31314 mini_ranger_kms.cc:401] 127.30.148.148:42127/kms/v1/keys
Jun 19 16:56:12 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781888172, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.148.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 16:56:12 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888172, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.148.148@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
Jun 19 16:56:12 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888172, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.148.148@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
Jun 19 16:56:15 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:56:15.187618 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:38601
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:38601
--ranger_config_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.148.148:42127/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:15.266945 11023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:15.267161 11023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:15.267194 11023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:15.268376 11023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:56:15.268440 11023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:15.268462 11023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:15.268491 11023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:56:15.268522 11023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:56:15.270076 11023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.148.148:42127/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:38601
--ranger_config_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:38601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:15.270315 11023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:15.270511 11023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:15.275699 11029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:15.275797 11028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:15.275770 11023 server_base.cc:1034] running on GCE node
W20260619 16:56:15.275694 11031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:15.276135 11023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:15.276459 11023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:15.277619 11023 hybrid_clock.cc:648] HybridClock initialized: now 1781888175277598 us; error 38 us; skew 500 ppm
Jun 19 16:56:15 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888175, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:15.283020 11023 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:56:15.283496 11023 webserver.cc:460] Webserver started at http://127.30.148.190:36429/ using document root <none> and password file <none>
I20260619 16:56:15.283721 11023 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:15.283771 11023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:15.283893 11023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 16:56:15 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888175, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:56:15.342655 11023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "e26a3e95403940b69ac27c8b5295af9f"
format_stamp: "Formatted at 2026-06-19 16:56:15 on dist-test-slave-xb9n"
server_key: "5abcc7d8a61336ef4cc450a16cf113a7"
server_key_iv: "8b20535d3e12e65f2ba43491945638c6"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:15.343048 11023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "e26a3e95403940b69ac27c8b5295af9f"
format_stamp: "Formatted at 2026-06-19 16:56:15 on dist-test-slave-xb9n"
server_key: "5abcc7d8a61336ef4cc450a16cf113a7"
server_key_iv: "8b20535d3e12e65f2ba43491945638c6"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:15.344466 11023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:56:15.359151 11038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:15.359326 11023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 16:56:15.359421 11023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "e26a3e95403940b69ac27c8b5295af9f"
format_stamp: "Formatted at 2026-06-19 16:56:15 on dist-test-slave-xb9n"
server_key: "5abcc7d8a61336ef4cc450a16cf113a7"
server_key_iv: "8b20535d3e12e65f2ba43491945638c6"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:15.359524 11023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:15.377249 11023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:15.379153 11023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:15.379313 11023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:15.384105 11023 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:38601
I20260619 16:56:15.384187 11090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:38601 every 8 connection(s)
I20260619 16:56:15.384541 11023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:56:15.385210 11091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:15.387440 11091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f: Bootstrap starting.
I20260619 16:56:15.388028 11091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:15.388337 11091 log.cc:824] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:15.388976 11091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f: No bootstrap required, opened a new log
I20260619 16:56:15.390974 11091 raft_consensus.cc:348] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26a3e95403940b69ac27c8b5295af9f" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38601 } }
I20260619 16:56:15.391135 11091 raft_consensus.cc:374] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:15.391197 11091 raft_consensus.cc:729] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e26a3e95403940b69ac27c8b5295af9f, State: Initialized, Role: FOLLOWER
I20260619 16:56:15.391363 11091 consensus_queue.cc:260] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26a3e95403940b69ac27c8b5295af9f" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38601 } }
I20260619 16:56:15.391427 11091 raft_consensus.cc:388] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:56:15.391448 11091 raft_consensus.cc:482] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:56:15.391508 11091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:15.392019 11091 raft_consensus.cc:504] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26a3e95403940b69ac27c8b5295af9f" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38601 } }
I20260619 16:56:15.392098 11091 leader_election.cc:302] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e26a3e95403940b69ac27c8b5295af9f; no voters:
I20260619 16:56:15.392261 11091 leader_election.cc:288] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:56:15.392310 11094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:56:15.392455 11094 raft_consensus.cc:686] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 1 LEADER]: Becoming Leader. State: Replica: e26a3e95403940b69ac27c8b5295af9f, State: Running, Role: LEADER
I20260619 16:56:15.392515 11091 sys_catalog.cc:564] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:56:15.392594 11094 consensus_queue.cc:237] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26a3e95403940b69ac27c8b5295af9f" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38601 } }
I20260619 16:56:15.393006 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 11023
I20260619 16:56:15.393103 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:56:15.393288 31314 external_mini_cluster.cc:1428] Setting key 7096edf28c391cc566ee7a8b46db398d
I20260619 16:56:15.394093 11091 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 16:56:15.394476 11095 sys_catalog.cc:455] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e26a3e95403940b69ac27c8b5295af9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26a3e95403940b69ac27c8b5295af9f" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38601 } } }
I20260619 16:56:15.394567 11095 sys_catalog.cc:458] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:15.396530 11096 sys_catalog.cc:455] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e26a3e95403940b69ac27c8b5295af9f. Latest consensus state: current_term: 1 leader_uuid: "e26a3e95403940b69ac27c8b5295af9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26a3e95403940b69ac27c8b5295af9f" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38601 } } }
I20260619 16:56:15.396813 11096 sys_catalog.cc:458] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:15.397235 11112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:56:15.398183 11112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 16:56:15 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:15.410724 11112 catalog_manager.cc:1347] Generated new cluster ID: 609628fbab1b4594a5e60e9af315ba09
I20260619 16:56:15.410789 11112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:56:15.412600 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:15.394033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.30.148.190:38601)
0619 16:56:15.394289 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:15.394300 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:15.394332 (+ 32us) server_negotiation.cc:407] Connection header received
0619 16:56:15.394456 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:15.394464 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:15.394525 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:15.394669 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:15.396755 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:15.398773 (+ 2018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:15.399924 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:15.400785 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:15.401209 (+ 424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:15.403688 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:15.403692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:15.403693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:15.403706 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:15.410413 (+ 6707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:15.410860 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:15.410863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:15.410864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:15.410978 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:15.411207 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:15.411209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:15.411209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:15.411549 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:15.412163 (+ 614us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:15.412236 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 16:56:15.412374 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":97,"threads_started":1}
I20260619 16:56:15.414759 11112 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:56:15.415459 11112 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:56:15.424247 11112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f: Generated new TSK 0
I20260619 16:56:15.424489 11112 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:56:16.129863 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.148.148:42127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38601
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:16.238967 11137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:16.239143 11137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:16.239166 11137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:16.240934 11137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:16.241074 11137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:16.241166 11137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:56:16.243309 11137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.148.148:42127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:16.243904 11137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:16.244159 11137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:16.249887 11144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:16.249984 11137 server_base.cc:1034] running on GCE node
W20260619 16:56:16.249872 11147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:16.250200 11145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:16.250435 11137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:16.250682 11137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:16.251839 11137 hybrid_clock.cc:648] HybridClock initialized: now 1781888176251829 us; error 30 us; skew 500 ppm
Jun 19 16:56:16 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:16.254943 11137 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:56:16.255456 11137 webserver.cc:460] Webserver started at http://127.30.148.129:40199/ using document root <none> and password file <none>
I20260619 16:56:16.255750 11137 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:16.255829 11137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:16.255995 11137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 16:56:16 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:56:16.270259 11137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "d185216317ed4c70b5754eb824efc7a9"
format_stamp: "Formatted at 2026-06-19 16:56:16 on dist-test-slave-xb9n"
server_key: "6f8552c42ee38be8679f116394e5b31d"
server_key_iv: "004185cc2c525f93dfdea4b8cdca00bf"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:16.270807 11137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "d185216317ed4c70b5754eb824efc7a9"
format_stamp: "Formatted at 2026-06-19 16:56:16 on dist-test-slave-xb9n"
server_key: "6f8552c42ee38be8679f116394e5b31d"
server_key_iv: "004185cc2c525f93dfdea4b8cdca00bf"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:16.272821 11137 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 16:56:16.292878 11154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:16.293115 11137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260619 16:56:16.293197 11137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "d185216317ed4c70b5754eb824efc7a9"
format_stamp: "Formatted at 2026-06-19 16:56:16 on dist-test-slave-xb9n"
server_key: "6f8552c42ee38be8679f116394e5b31d"
server_key_iv: "004185cc2c525f93dfdea4b8cdca00bf"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:16.293278 11137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:16.314374 11137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:16.316747 11137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:16.316939 11137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:16.317193 11137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:16.317591 11137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:16.317665 11137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:16.317728 11137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:16.317775 11137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:16.331111 11137 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:42939
I20260619 16:56:16.331514 11137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:56:16.333950 11267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:42939 every 8 connection(s)
I20260619 16:56:16.335923 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 11137
I20260619 16:56:16.336032 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:56:16.336229 31314 external_mini_cluster.cc:1428] Setting key 45af78ee04c9a1c24db53b49becf9937
Jun 19 16:56:16 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
I20260619 16:56:16.347208 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:16.334768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:33147 (local address 127.30.148.190:38601)
0619 16:56:16.336443 (+ 1675us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:16.336445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:16.337374 (+ 929us) server_negotiation.cc:407] Connection header received
0619 16:56:16.337443 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:16.337444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:16.337457 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:16.337517 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:16.338262 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:16.339043 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:16.340558 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:16.340900 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:16.340987 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:16.343629 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:16.343630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:16.343630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:16.343646 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:16.345723 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:16.346193 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:16.346194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:16.346195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:16.346257 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:16.346506 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:16.346507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:16.346508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:16.346621 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:16.346694 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:16.347030 (+ 336us) server_negotiation.cc:299] Negotiation successful
0619 16:56:16.347111 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1636,"thread_start_us":79,"threads_started":1}
I20260619 16:56:16.347409 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:16.334731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:38601 (local address 127.30.148.129:33147)
0619 16:56:16.336516 (+ 1785us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:16.336562 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:16.337241 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:16.337640 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:16.337644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:16.337887 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:16.338151 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:16.338157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:16.339152 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:16.339153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:16.340443 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:16.340446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:16.341073 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:16.341074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:16.341215 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:16.341865 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:16.341875 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:16.343500 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:16.345842 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:16.345849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:16.345851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:16.346116 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:16.346324 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:16.346325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:16.346325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:16.346435 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:16.346685 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:16.346691 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:16.346896 (+ 205us) client_negotiation.cc:769] Sending connection context
0619 16:56:16.347009 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 16:56:16.347148 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1740,"thread_start_us":80,"threads_started":1}
I20260619 16:56:16.348045 11268 heartbeater.cc:347] Connected to a master server at 127.30.148.190:38601
I20260619 16:56:16.348156 11268 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:16.348400 11268 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
I20260619 16:56:16.348932 11055 ts_manager.cc:194] Registered new tserver with Master: d185216317ed4c70b5754eb824efc7a9 (127.30.148.129:42939)
I20260619 16:56:16.349934 11055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:33147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:56:16.366577 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.148.148:42127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38601
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 16:56:16 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781888176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 16:56:16.465126 11276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:16.465452 11276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:16.465509 11276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:16.466708 11276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:16.466778 11276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:16.466822 11276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:56:16.468290 11276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.148.148:42127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:16.468540 11276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:16.468770 11276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:16.473655 11282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:16.473764 11281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:16.473634 11284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:16.474233 11276 server_base.cc:1034] running on GCE node
I20260619 16:56:16.474432 11276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:16.474628 11276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:16.475770 11276 hybrid_clock.cc:648] HybridClock initialized: now 1781888176475751 us; error 36 us; skew 500 ppm
Jun 19 16:56:16 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:16.478317 11276 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:56:16.478781 11276 webserver.cc:460] Webserver started at http://127.30.148.130:39935/ using document root <none> and password file <none>
I20260619 16:56:16.479007 11276 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:16.479068 11276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:16.479197 11276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 16:56:16 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:56:16.492277 11276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7"
format_stamp: "Formatted at 2026-06-19 16:56:16 on dist-test-slave-xb9n"
server_key: "6980711ab9293e32033f0e2843a7f53b"
server_key_iv: "90613ee843902b51ede582161eec9932"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:16.492705 11276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7"
format_stamp: "Formatted at 2026-06-19 16:56:16 on dist-test-slave-xb9n"
server_key: "6980711ab9293e32033f0e2843a7f53b"
server_key_iv: "90613ee843902b51ede582161eec9932"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:16.494488 11276 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260619 16:56:16.512676 11292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:16.512908 11276 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260619 16:56:16.512993 11276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7"
format_stamp: "Formatted at 2026-06-19 16:56:16 on dist-test-slave-xb9n"
server_key: "6980711ab9293e32033f0e2843a7f53b"
server_key_iv: "90613ee843902b51ede582161eec9932"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:16.513074 11276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:16.547919 11276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:16.550391 11276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:16.550606 11276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:16.550859 11276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:16.551301 11276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:16.551403 11276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:16.551486 11276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:16.551546 11276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:16.559580 11276 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:37315
I20260619 16:56:16.560034 11276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:56:16.561568 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 11276
I20260619 16:56:16.561667 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:56:16.561861 31314 external_mini_cluster.cc:1428] Setting key 43aa5b309303141829152402698ddf11
I20260619 16:56:16.560762 11405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:37315 every 8 connection(s)
Jun 19 16:56:16 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:16.572350 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:16.561856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:46283 (local address 127.30.148.190:38601)
0619 16:56:16.561937 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:16.561939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:16.562303 (+ 364us) server_negotiation.cc:407] Connection header received
0619 16:56:16.563074 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:16.563075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:16.563091 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:16.563122 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:16.563934 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:16.564711 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:16.566316 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:16.566629 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:16.566696 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:16.569128 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:16.569129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:16.569130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:16.569137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:16.570837 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:16.571333 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:16.571334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:16.571335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:16.571393 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:16.571657 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:16.571658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:16.571659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:16.571783 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:16.571824 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:16.572157 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 16:56:16.572240 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 16:56:16.572831 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:16.561537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:38601 (local address 127.30.148.130:46283)
0619 16:56:16.562160 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:16.562202 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:16.562949 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:16.563323 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:16.563327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:16.563552 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:16.563820 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:16.563828 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:16.564832 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:16.564833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:16.566200 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:16.566203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:16.566771 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:16.566773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:16.566914 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:16.567481 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:16.567491 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:16.569020 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:16.570950 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:16.570957 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:16.570959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:16.571231 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:16.571464 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:16.571464 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:16.571465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:16.571588 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:16.571868 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:16.571873 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:16.572020 (+ 147us) client_negotiation.cc:769] Sending connection context
0619 16:56:16.572470 (+ 450us) client_negotiation.cc:240] Negotiation successful
0619 16:56:16.572618 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":89,"threads_started":1}
I20260619 16:56:16.573449 11406 heartbeater.cc:347] Connected to a master server at 127.30.148.190:38601
I20260619 16:56:16.573549 11406 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:16.573731 11406 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
I20260619 16:56:16.574100 11055 ts_manager.cc:194] Registered new tserver with Master: 257b6e53ce2248c0b7c7a6c0a799d1a7 (127.30.148.130:37315)
I20260619 16:56:16.574626 11055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:46283
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:56:16.602068 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.148.148:42127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38601
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:16.717919 11411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:16.718148 11411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:16.718185 11411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:16.719401 11411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:16.719484 11411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:16.719539 11411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:56:16.721635 11411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.148.148:42127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:16.721983 11411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:16.722251 11411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:16.728950 11419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:16.729326 11411 server_base.cc:1034] running on GCE node
W20260619 16:56:16.730391 11416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:16.730643 11417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:16.730881 11411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:16.731178 11411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:16.732329 11411 hybrid_clock.cc:648] HybridClock initialized: now 1781888176732319 us; error 30 us; skew 500 ppm
Jun 19 16:56:16 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:16.734899 11411 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:56:16.735404 11411 webserver.cc:460] Webserver started at http://127.30.148.131:41641/ using document root <none> and password file <none>
I20260619 16:56:16.735637 11411 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:16.735687 11411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:16.735822 11411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 16:56:16 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:56:16.752806 11411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "6d8c96cedcfb4508a68f7e68e175a843"
format_stamp: "Formatted at 2026-06-19 16:56:16 on dist-test-slave-xb9n"
server_key: "467baff31e6f25ccd64fc47510e7c6b3"
server_key_iv: "d282faa5b66e7668fd9e2d55df17cb59"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:16.753226 11411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "6d8c96cedcfb4508a68f7e68e175a843"
format_stamp: "Formatted at 2026-06-19 16:56:16 on dist-test-slave-xb9n"
server_key: "467baff31e6f25ccd64fc47510e7c6b3"
server_key_iv: "d282faa5b66e7668fd9e2d55df17cb59"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:16.762503 11411 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.000s
I20260619 16:56:16.778918 11426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:16.779084 11411 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260619 16:56:16.779174 11411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "6d8c96cedcfb4508a68f7e68e175a843"
format_stamp: "Formatted at 2026-06-19 16:56:16 on dist-test-slave-xb9n"
server_key: "467baff31e6f25ccd64fc47510e7c6b3"
server_key_iv: "d282faa5b66e7668fd9e2d55df17cb59"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:16.779264 11411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:16.814906 11411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:16.817317 11411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:16.817521 11411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:16.817735 11411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:16.818078 11411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:16.818132 11411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:16.818176 11411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:16.818202 11411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:16.824937 11411 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:41187
I20260619 16:56:16.825394 11411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:56:16.826745 11539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:41187 every 8 connection(s)
I20260619 16:56:16.829759 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 11411
I20260619 16:56:16.829861 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:56:16.830044 31314 external_mini_cluster.cc:1428] Setting key 6c5185d934450fe6fc65ee5f3acdec99
Jun 19 16:56:16 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:16.838362 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:16.826412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:44017 (local address 127.30.148.190:38601)
0619 16:56:16.826455 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:16.826457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:16.826817 (+ 360us) server_negotiation.cc:407] Connection header received
0619 16:56:16.827568 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:16.827570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:16.827583 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:16.827614 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:16.828434 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:16.829249 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:16.832164 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:16.832483 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:16.832568 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:16.835074 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:16.835076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:16.835076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:16.835084 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:16.836792 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:16.837254 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:16.837256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:16.837256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:16.837326 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:16.837658 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:16.837659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:16.837659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:16.837794 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:16.837836 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:16.838178 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 16:56:16.838258 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:56:16.838850 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:16.826395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:38601 (local address 127.30.148.131:44017)
0619 16:56:16.826680 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:16.826728 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:16.827420 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:16.827810 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:16.827814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:16.828047 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:16.828316 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:16.828322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:16.829414 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:16.829415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:16.832060 (+ 2645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:16.832064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:16.832588 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:16.832589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:16.832739 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:16.833322 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:16.833332 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:16.834982 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:16.836881 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:16.836887 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:16.836889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:16.837171 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:16.837437 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:16.837438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:16.837438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:16.837571 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:16.837882 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:16.837887 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:16.838037 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 16:56:16.838485 (+ 448us) client_negotiation.cc:240] Negotiation successful
0619 16:56:16.838634 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":188,"threads_started":1}
I20260619 16:56:16.839231 11540 heartbeater.cc:347] Connected to a master server at 127.30.148.190:38601
I20260619 16:56:16.839331 11540 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:16.839507 11540 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
I20260619 16:56:16.839818 11055 ts_manager.cc:194] Registered new tserver with Master: 6d8c96cedcfb4508a68f7e68e175a843 (127.30.148.131:41187)
I20260619 16:56:16.840309 11055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:44017
I20260619 16:56:16.841575 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:56:16.842242 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 11137
I20260619 16:56:16.849017 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 11276
I20260619 16:56:16.854514 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 11411
I20260619 16:56:16.859759 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 11023
I20260619 16:56:16.866062 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:38601
--webserver_interface=127.30.148.190
--webserver_port=36429
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:38601
--ranger_config_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.148.148:42127/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.148.148:42127/kms with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:16.944689 11543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:16.944883 11543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:16.944904 11543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:16.946017 11543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:56:16.946058 11543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:16.946071 11543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:16.946103 11543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:56:16.946132 11543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:56:16.947603 11543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.148.148:42127/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:38601
--ranger_config_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:38601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=36429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:16.947844 11543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:16.948048 11543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:16.952739 11549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:16.952844 11543 server_base.cc:1034] running on GCE node
W20260619 16:56:16.952737 11551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:16.952831 11548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:16.953156 11543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:16.953428 11543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:16.954582 11543 hybrid_clock.cc:648] HybridClock initialized: now 1781888176954566 us; error 29 us; skew 500 ppm
Jun 19 16:56:16 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:16.956988 11543 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:56:16.957445 11543 webserver.cc:460] Webserver started at http://127.30.148.190:36429/ using document root <none> and password file <none>
I20260619 16:56:16.957608 11543 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:16.957654 11543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 16:56:16 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888176, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:56:16.993562 11543 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:56:16.994307 11559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:16.994453 11543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:56:16.994553 11543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "e26a3e95403940b69ac27c8b5295af9f"
format_stamp: "Formatted at 2026-06-19 16:56:15 on dist-test-slave-xb9n"
server_key: "5abcc7d8a61336ef4cc450a16cf113a7"
server_key_iv: "8b20535d3e12e65f2ba43491945638c6"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:16.994899 11543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:17.004473 11543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:17.006016 11543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:17.006155 11543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:17.010166 11543 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:38601
I20260619 16:56:17.010205 11611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:38601 every 8 connection(s)
I20260619 16:56:17.010550 11543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:56:17.010758 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 11543
I20260619 16:56:17.011068 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.148.148:42127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:42939
--local_ip_for_outbound_sockets=127.30.148.129
--tserver_master_addrs=127.30.148.190:38601
--webserver_port=40199
--webserver_interface=127.30.148.129
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.148.148:42127/kms with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:56:17.011755 11612 sys_catalog.cc:263] Verifying existing consensus state
I20260619 16:56:17.012364 11612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f: Bootstrap starting.
I20260619 16:56:17.022744 11612 log.cc:824] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:17.024660 11612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:56:17.024941 11612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f: Bootstrap complete.
I20260619 16:56:17.028230 11612 raft_consensus.cc:348] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26a3e95403940b69ac27c8b5295af9f" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38601 } }
I20260619 16:56:17.028537 11612 raft_consensus.cc:729] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e26a3e95403940b69ac27c8b5295af9f, State: Initialized, Role: FOLLOWER
I20260619 16:56:17.028683 11612 consensus_queue.cc:260] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26a3e95403940b69ac27c8b5295af9f" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38601 } }
I20260619 16:56:17.028765 11612 raft_consensus.cc:388] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:56:17.028803 11612 raft_consensus.cc:482] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:56:17.028847 11612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 1 FOLLOWER]: Advancing to term 2
I20260619 16:56:17.030085 11612 raft_consensus.cc:504] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26a3e95403940b69ac27c8b5295af9f" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38601 } }
I20260619 16:56:17.030185 11612 leader_election.cc:302] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e26a3e95403940b69ac27c8b5295af9f; no voters:
I20260619 16:56:17.030354 11612 leader_election.cc:288] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 16:56:17.030422 11616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 2 FOLLOWER]: Leader election won for term 2
I20260619 16:56:17.030565 11616 raft_consensus.cc:686] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [term 2 LEADER]: Becoming Leader. State: Replica: e26a3e95403940b69ac27c8b5295af9f, State: Running, Role: LEADER
I20260619 16:56:17.030611 11612 sys_catalog.cc:564] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:56:17.030673 11616 consensus_queue.cc:237] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26a3e95403940b69ac27c8b5295af9f" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38601 } }
I20260619 16:56:17.031029 11617 sys_catalog.cc:455] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e26a3e95403940b69ac27c8b5295af9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26a3e95403940b69ac27c8b5295af9f" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38601 } } }
I20260619 16:56:17.031104 11617 sys_catalog.cc:458] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:17.031322 11618 sys_catalog.cc:455] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e26a3e95403940b69ac27c8b5295af9f. Latest consensus state: current_term: 2 leader_uuid: "e26a3e95403940b69ac27c8b5295af9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26a3e95403940b69ac27c8b5295af9f" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38601 } } }
I20260619 16:56:17.031443 11623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:56:17.031446 11618 sys_catalog.cc:458] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:17.031951 11623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:56:17.032073 11623 catalog_manager.cc:1259] Loaded cluster ID: 609628fbab1b4594a5e60e9af315ba09
I20260619 16:56:17.032111 11623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:56:17.032285 11612 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 16:56:17.035830 11623 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:56:17.036381 11623 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f: Loaded TSK: 0
I20260619 16:56:17.036669 11623 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 16:56:17.127238 11614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:17.127504 11614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:17.127537 11614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:17.129253 11614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:17.129314 11614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:17.129371 11614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:56:17.131466 11614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.148.148:42127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:42939
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=40199
--tserver_master_addrs=127.30.148.190:38601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:17.131685 11614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:17.131912 11614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:17.140568 11650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:17.140563 11652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:17.141417 11649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:17.143887 11614 server_base.cc:1034] running on GCE node
I20260619 16:56:17.144101 11614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:17.144376 11614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:17.145545 11614 hybrid_clock.cc:648] HybridClock initialized: now 1781888177145518 us; error 42 us; skew 500 ppm
Jun 19 16:56:17 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888177, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:17.148761 11614 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:56:17.149305 11614 webserver.cc:460] Webserver started at http://127.30.148.129:40199/ using document root <none> and password file <none>
I20260619 16:56:17.149603 11614 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:17.149688 11614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 16:56:17 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888177, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:56:17.172669 11614 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260619 16:56:17.173674 11660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:17.173884 11614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 16:56:17.173974 11614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "d185216317ed4c70b5754eb824efc7a9"
format_stamp: "Formatted at 2026-06-19 16:56:16 on dist-test-slave-xb9n"
server_key: "6f8552c42ee38be8679f116394e5b31d"
server_key_iv: "004185cc2c525f93dfdea4b8cdca00bf"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:17.174352 11614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:17.191195 11614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:17.193406 11614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:17.193557 11614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:17.193796 11614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:17.194144 11614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:17.194193 11614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:17.194229 11614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:17.194253 11614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:17.200896 11614 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:42939
I20260619 16:56:17.201227 11614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:56:17.201769 11773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:42939 every 8 connection(s)
I20260619 16:56:17.207432 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 11614
I20260619 16:56:17.207773 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.148.148:42127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:37315
--local_ip_for_outbound_sockets=127.30.148.130
--tserver_master_addrs=127.30.148.190:38601
--webserver_port=39935
--webserver_interface=127.30.148.130
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.148.148:42127/kms with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 16:56:17 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888177, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:17.212658 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:17.202425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:38601 (local address 127.30.148.129:41831)
0619 16:56:17.202592 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:17.202638 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:17.203114 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:17.203558 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:17.203562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:17.203770 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:17.203986 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:17.203991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.205107 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.205108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:17.205926 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:17.205928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.206521 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.206522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:17.206636 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:17.207145 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:17.207152 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:17.208494 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:17.210761 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:17.210767 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:17.210770 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:17.211073 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:17.211355 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:17.211356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:17.211356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:17.211485 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:17.211993 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:17.211999 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:17.212149 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 16:56:17.212242 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 16:56:17.212388 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260619 16:56:17.213729 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:17.202492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:41831 (local address 127.30.148.190:38601)
0619 16:56:17.202702 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:17.202709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:17.202743 (+ 34us) server_negotiation.cc:407] Connection header received
0619 16:56:17.203348 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:17.203353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:17.203394 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:17.203512 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:17.204090 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.205005 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.206012 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.206406 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.206531 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:17.208597 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:17.208601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:17.208602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:17.208613 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:17.210668 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:17.211168 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:17.211171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:17.211172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:17.211280 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:17.211557 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:17.211558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:17.211558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:17.211908 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:17.213242 (+ 1334us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:17.213322 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 16:56:17.213488 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
I20260619 16:56:17.214560 11774 heartbeater.cc:347] Connected to a master server at 127.30.148.190:38601
I20260619 16:56:17.214641 11774 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:17.214803 11774 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
W20260619 16:56:17.215673 11774 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:38601 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 16:56:17.215749 11774 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:17.215808 11774 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
I20260619 16:56:17.216174 11774 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:17.216253 11774 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
W20260619 16:56:17.216579 11774 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 16:56:17.316552 11778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:17.316816 11778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:17.316859 11778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:17.318641 11778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:17.318698 11778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:17.318751 11778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:56:17.320737 11778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.148.148:42127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:37315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=39935
--tserver_master_addrs=127.30.148.190:38601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:17.320967 11778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:17.321187 11778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:17.331868 11786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:17.332305 11784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:17.332427 11783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:17.332760 11778 server_base.cc:1034] running on GCE node
I20260619 16:56:17.333042 11778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:17.333395 11778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:17.343428 11778 hybrid_clock.cc:648] HybridClock initialized: now 1781888177343399 us; error 47 us; skew 500 ppm
Jun 19 16:56:17 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888177, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:17.351081 11778 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:56:17.351598 11778 webserver.cc:460] Webserver started at http://127.30.148.130:39935/ using document root <none> and password file <none>
I20260619 16:56:17.351986 11778 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:17.352103 11778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 16:56:17 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888177, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:56:17.368943 11778 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:17.373718 11794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:17.373909 11778 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260619 16:56:17.374017 11778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7"
format_stamp: "Formatted at 2026-06-19 16:56:16 on dist-test-slave-xb9n"
server_key: "6980711ab9293e32033f0e2843a7f53b"
server_key_iv: "90613ee843902b51ede582161eec9932"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:17.374503 11778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:17.385146 11778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:17.387632 11778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:17.387799 11778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:17.388038 11778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:17.388361 11778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:17.388394 11778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:17.388442 11778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:17.388478 11778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:17.395709 11778 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:37315
I20260619 16:56:17.395839 11907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:37315 every 8 connection(s)
I20260619 16:56:17.396131 11778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:56:17.399269 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 11778
I20260619 16:56:17.399519 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.148.148:42127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:41187
--local_ip_for_outbound_sockets=127.30.148.131
--tserver_master_addrs=127.30.148.190:38601
--webserver_port=41641
--webserver_interface=127.30.148.131
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.148.148:42127/kms with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 16:56:17 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888177, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:17.406999 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:17.397002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:38701 (local address 127.30.148.190:38601)
0619 16:56:17.397061 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:17.397063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:17.397266 (+ 203us) server_negotiation.cc:407] Connection header received
0619 16:56:17.397841 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:17.397843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:17.397859 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:17.397935 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:17.398567 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.399340 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.400850 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.401175 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.401252 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:17.403653 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:17.403654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:17.403655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:17.403662 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:17.405333 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:17.405860 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:17.405861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:17.405862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:17.405924 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:17.406213 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:17.406214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:17.406214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:17.406339 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:17.406387 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:17.406775 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 16:56:17.406870 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 16:56:17.407085 11910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:17.396996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:38601 (local address 127.30.148.130:38701)
0619 16:56:17.397162 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:17.397196 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:17.397735 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:17.397984 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:17.397987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:17.398214 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:17.398451 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:17.398456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.399457 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.399458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:17.400748 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:17.400752 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.401295 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.401297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:17.401479 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:17.401999 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:17.402011 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:17.403540 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:17.405470 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:17.405477 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:17.405479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:17.405765 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:17.406005 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:17.406006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:17.406007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:17.406134 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:17.406442 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:17.406448 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:17.406611 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 16:56:17.406723 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 16:56:17.406862 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":103,"threads_started":1}
I20260619 16:56:17.407624 11908 heartbeater.cc:347] Connected to a master server at 127.30.148.190:38601
I20260619 16:56:17.407734 11908 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:17.407927 11908 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
W20260619 16:56:17.408643 11908 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:38601 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 16:56:17.408715 11908 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:17.408782 11908 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
I20260619 16:56:17.409137 11908 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:17.409224 11908 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
W20260619 16:56:17.409632 11908 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 16:56:17.495277 11911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:17.495486 11911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:17.495523 11911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:17.496619 11911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:17.496677 11911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:17.496719 11911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:56:17.498771 11911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.148.148:42127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:41187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=41641
--tserver_master_addrs=127.30.148.190:38601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:17.498999 11911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:17.499223 11911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:17.504078 11916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:17.504155 11917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:17.504487 11919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:17.505056 11911 server_base.cc:1034] running on GCE node
I20260619 16:56:17.505249 11911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:17.505508 11911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:17.506659 11911 hybrid_clock.cc:648] HybridClock initialized: now 1781888177506629 us; error 41 us; skew 500 ppm
Jun 19 16:56:17 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888177, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:17.509172 11911 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:56:17.509680 11911 webserver.cc:460] Webserver started at http://127.30.148.131:41641/ using document root <none> and password file <none>
I20260619 16:56:17.509917 11911 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:17.509991 11911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 16:56:17 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888177, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:56:17.529726 11911 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 16:56:17.532409 11928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:17.532646 11911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260619 16:56:17.532747 11911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "6d8c96cedcfb4508a68f7e68e175a843"
format_stamp: "Formatted at 2026-06-19 16:56:16 on dist-test-slave-xb9n"
server_key: "467baff31e6f25ccd64fc47510e7c6b3"
server_key_iv: "d282faa5b66e7668fd9e2d55df17cb59"
server_key_version: "kuduclusterkey@0"
I20260619 16:56:17.533177 11911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:17.549472 11911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:17.551577 11911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:17.551719 11911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:17.551955 11911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:17.552248 11911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:17.552294 11911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:17.552328 11911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:17.552354 11911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:17.560745 11911 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:41187
I20260619 16:56:17.560827 12041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:41187 every 8 connection(s)
I20260619 16:56:17.561187 11911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:56:17.565032 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 11911
Jun 19 16:56:17 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888177, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:17.572197 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:17.562200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:59051 (local address 127.30.148.190:38601)
0619 16:56:17.562248 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:17.562251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:17.562261 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:56:17.562841 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:17.562843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:17.562854 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:17.562913 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:17.563514 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.564256 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.565761 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.566079 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.566141 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:17.569019 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:17.569021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:17.569021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:17.569028 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:17.570864 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:17.571241 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:17.571242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:17.571242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:17.571297 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:17.571539 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:17.571540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:17.571540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:17.571668 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:17.571729 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:17.572031 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 16:56:17.572114 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 16:56:17.572386 12044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:17.561929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:38601 (local address 127.30.148.131:59051)
0619 16:56:17.562109 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:17.562153 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:17.562731 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:17.562975 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:17.562979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:17.563170 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:17.563429 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:17.563435 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.564347 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.564348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:17.565661 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:17.565664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.566220 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.566222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:17.566358 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:17.566802 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:17.566811 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:17.568913 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:17.570937 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:17.570943 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:17.570944 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:17.571153 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:17.571365 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:17.571366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:17.571366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:17.571478 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:17.571756 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:17.571762 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:17.571899 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 16:56:17.572009 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 16:56:17.572143 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
I20260619 16:56:17.572870 12042 heartbeater.cc:347] Connected to a master server at 127.30.148.190:38601
I20260619 16:56:17.572975 12042 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:17.573164 12042 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
W20260619 16:56:17.573868 12042 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:38601 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 16:56:17.573944 12042 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:17.574018 12042 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
I20260619 16:56:17.574328 12042 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:17.574407 12042 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
W20260619 16:56:17.574724 12042 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 16:56:17.578365 12046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:17.565443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58906 (local address 127.30.148.190:38601)
0619 16:56:17.565956 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:17.565958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:17.565965 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:56:17.565986 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:17.565987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:17.565997 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:17.566995 (+ 998us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:17.567025 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.567577 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.568673 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:17.568870 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:17.568943 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:17.570308 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:17.570309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:17.570310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:17.570317 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:17.572516 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:17.573137 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:17.573138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:17.573138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:17.573196 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:17.575754 (+ 2558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:17.575755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:17.575755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:17.575891 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:17.576054 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:17.578177 (+ 2123us) server_negotiation.cc:299] Negotiation successful
0619 16:56:17.578267 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":70,"threads_started":1}
Jun 19 16:56:18 dist-test-slave-xb9n krb5kdc[10579](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781888178, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:18.216766 11774 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:18.216917 11774 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
I20260619 16:56:18.217325 11576 ts_manager.cc:194] Registered new tserver with Master: d185216317ed4c70b5754eb824efc7a9 (127.30.148.129:42939)
I20260619 16:56:18.218340 11576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:41831
I20260619 16:56:18.409811 11908 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:18.409955 11908 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
I20260619 16:56:18.410259 11576 ts_manager.cc:194] Registered new tserver with Master: 257b6e53ce2248c0b7c7a6c0a799d1a7 (127.30.148.130:37315)
I20260619 16:56:18.410817 11576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:38701
Jun 19 16:56:18 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888178, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:56:18.574887 12042 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:18.575024 12042 heartbeater.cc:511] Master 127.30.148.190:38601 requested a full tablet report, sending...
I20260619 16:56:18.575310 11576 ts_manager.cc:194] Registered new tserver with Master: 6d8c96cedcfb4508a68f7e68e175a843 (127.30.148.131:41187)
I20260619 16:56:18.575460 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:56:18.575838 11576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:59051
I20260619 16:56:18.584918 12063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.576463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58922 (local address 127.30.148.190:38601)
0619 16:56:18.576722 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:18.576724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:18.576824 (+ 100us) server_negotiation.cc:407] Connection header received
0619 16:56:18.576917 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:18.576918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:18.576928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:18.576971 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:18.577779 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.578680 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.580067 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.580371 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.580458 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:18.581990 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:18.581992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:18.581992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:18.581999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:18.583722 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:18.584105 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:18.584106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:18.584107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:18.584174 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:18.584390 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:18.584390 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:18.584391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:18.584524 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:18.584626 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:18.584745 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 16:56:18.584822 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":58,"threads_started":1}
I20260619 16:56:18.587060 11576 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:56:18.589224 11576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:56:18.597078 12072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.592078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:37315 (local address 127.0.0.1:51884)
0619 16:56:18.592388 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:18.592422 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:18.592505 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:18.592765 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:18.592769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:18.592770 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:18.592823 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:18.592824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.594063 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.594064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:18.595567 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:18.595570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.596550 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.596551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:18.596630 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:18.596642 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 16:56:18.596940 (+ 298us) client_negotiation.cc:240] Negotiation successful
0619 16:56:18.597008 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":81,"threads_started":1}
I20260619 16:56:18.597417 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.592446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44544 (local address 127.30.148.131:41187)
0619 16:56:18.593480 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:18.593484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:18.593638 (+ 154us) server_negotiation.cc:407] Connection header received
0619 16:56:18.593667 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:18.593669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:18.593682 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:18.593758 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:18.593947 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.594924 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.596154 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.596872 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.596946 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:18.597001 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:18.597292 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 16:56:18.597327 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1004,"thread_start_us":44,"threads_started":1}
I20260619 16:56:18.597878 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.592170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51884 (local address 127.30.148.130:37315)
0619 16:56:18.592305 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:18.592310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:18.592474 (+ 164us) server_negotiation.cc:407] Connection header received
0619 16:56:18.592583 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:18.592586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:18.592601 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:18.592678 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:18.592898 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.593976 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.595671 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.596396 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.596465 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:18.596516 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:18.597760 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0619 16:56:18.597803 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":55,"threads_started":1}
I20260619 16:56:18.597236 12070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.592082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:41187 (local address 127.0.0.1:44544)
0619 16:56:18.593532 (+ 1450us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:18.593541 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:18.593590 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:18.593831 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:18.593832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:18.593833 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:18.593879 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:18.593880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.595014 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.595014 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:18.596055 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:18.596058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.597032 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.597033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:18.597093 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:18.597097 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:56:18.597139 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 16:56:18.597173 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1434,"thread_start_us":95,"threads_started":1}
I20260619 16:56:18.598666 11824 tablet_service.cc:1467] Processing CreateTablet for tablet 54853a77ebc24e46b60d2c02569d0de1 (DEFAULT_TABLE table=test-table [id=d6d77fc9c428445f8e4ac6eb5f907c17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:56:18.599031 11824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54853a77ebc24e46b60d2c02569d0de1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:18.601238 12072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.592286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:42939 (local address 127.0.0.1:58460)
0619 16:56:18.597463 (+ 5177us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:18.597610 (+ 147us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:18.597687 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:18.597938 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:18.597939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:18.597940 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:18.597968 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:18.597969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.599131 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.599131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:18.600140 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:18.600143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.601014 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.601015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:18.601072 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:18.601075 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:56:18.601142 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 16:56:18.601181 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5168,"thread_start_us":61,"threads_started":1}
I20260619 16:56:18.601271 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.592369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58460 (local address 127.30.148.129:42939)
0619 16:56:18.592508 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:18.592512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:18.597705 (+ 5193us) server_negotiation.cc:407] Connection header received
0619 16:56:18.597750 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:18.597752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:18.597768 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:18.597849 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:18.598042 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.599033 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.600219 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.600920 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.601002 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:18.601049 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:18.601154 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 16:56:18.601189 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":51,"threads_started":1}
I20260619 16:56:18.602027 12075 tablet_bootstrap.cc:492] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7: Bootstrap starting.
I20260619 16:56:18.602661 12075 tablet_bootstrap.cc:654] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:18.602828 11708 tablet_service.cc:1467] Processing CreateTablet for tablet 54853a77ebc24e46b60d2c02569d0de1 (DEFAULT_TABLE table=test-table [id=d6d77fc9c428445f8e4ac6eb5f907c17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:56:18.602851 11976 tablet_service.cc:1467] Processing CreateTablet for tablet 54853a77ebc24e46b60d2c02569d0de1 (DEFAULT_TABLE table=test-table [id=d6d77fc9c428445f8e4ac6eb5f907c17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:56:18.602964 12075 log.cc:824] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:18.603143 11976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54853a77ebc24e46b60d2c02569d0de1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:18.603143 11708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54853a77ebc24e46b60d2c02569d0de1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:18.605388 12077 tablet_bootstrap.cc:492] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9: Bootstrap starting.
I20260619 16:56:18.605918 12077 tablet_bootstrap.cc:654] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:18.606205 12077 log.cc:824] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:18.606910 12077 tablet_bootstrap.cc:492] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9: No bootstrap required, opened a new log
I20260619 16:56:18.606983 12077 ts_tablet_manager.cc:1397] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:56:18.609141 12077 raft_consensus.cc:348] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 37315 } } peers { permanent_uuid: "6d8c96cedcfb4508a68f7e68e175a843" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41187 } } peers { permanent_uuid: "d185216317ed4c70b5754eb824efc7a9" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42939 } }
I20260619 16:56:18.609305 12077 raft_consensus.cc:374] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:18.609373 12077 raft_consensus.cc:729] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d185216317ed4c70b5754eb824efc7a9, State: Initialized, Role: FOLLOWER
I20260619 16:56:18.609491 12077 consensus_queue.cc:260] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 37315 } } peers { permanent_uuid: "6d8c96cedcfb4508a68f7e68e175a843" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41187 } } peers { permanent_uuid: "d185216317ed4c70b5754eb824efc7a9" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42939 } }
I20260619 16:56:18.609717 12077 ts_tablet_manager.cc:1428] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:56:18.609860 11774 heartbeater.cc:503] Master 127.30.148.190:38601 was elected leader, sending a full tablet report...
I20260619 16:56:18.610385 12075 tablet_bootstrap.cc:492] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7: No bootstrap required, opened a new log
I20260619 16:56:18.610469 12075 ts_tablet_manager.cc:1397] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260619 16:56:18.612920 12075 raft_consensus.cc:348] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 37315 } } peers { permanent_uuid: "6d8c96cedcfb4508a68f7e68e175a843" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41187 } } peers { permanent_uuid: "d185216317ed4c70b5754eb824efc7a9" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42939 } }
I20260619 16:56:18.613072 12075 raft_consensus.cc:374] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:18.613107 12075 raft_consensus.cc:729] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257b6e53ce2248c0b7c7a6c0a799d1a7, State: Initialized, Role: FOLLOWER
I20260619 16:56:18.613212 12075 consensus_queue.cc:260] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 37315 } } peers { permanent_uuid: "6d8c96cedcfb4508a68f7e68e175a843" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41187 } } peers { permanent_uuid: "d185216317ed4c70b5754eb824efc7a9" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42939 } }
I20260619 16:56:18.613490 12075 ts_tablet_manager.cc:1428] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 16:56:18.613606 11908 heartbeater.cc:503] Master 127.30.148.190:38601 was elected leader, sending a full tablet report...
I20260619 16:56:18.613492 12079 tablet_bootstrap.cc:492] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843: Bootstrap starting.
I20260619 16:56:18.614902 12079 tablet_bootstrap.cc:654] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:18.615199 12079 log.cc:824] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:18.615942 12079 tablet_bootstrap.cc:492] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843: No bootstrap required, opened a new log
I20260619 16:56:18.616006 12079 ts_tablet_manager.cc:1397] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 16:56:18.618470 12079 raft_consensus.cc:348] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 37315 } } peers { permanent_uuid: "6d8c96cedcfb4508a68f7e68e175a843" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41187 } } peers { permanent_uuid: "d185216317ed4c70b5754eb824efc7a9" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42939 } }
I20260619 16:56:18.618620 12079 raft_consensus.cc:374] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:18.618652 12079 raft_consensus.cc:729] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d8c96cedcfb4508a68f7e68e175a843, State: Initialized, Role: FOLLOWER
I20260619 16:56:18.618769 12079 consensus_queue.cc:260] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 37315 } } peers { permanent_uuid: "6d8c96cedcfb4508a68f7e68e175a843" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41187 } } peers { permanent_uuid: "d185216317ed4c70b5754eb824efc7a9" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42939 } }
I20260619 16:56:18.618992 12079 ts_tablet_manager.cc:1428] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:56:18.619022 12042 heartbeater.cc:503] Master 127.30.148.190:38601 was elected leader, sending a full tablet report...
W20260619 16:56:18.647558 11909 tablet.cc:2369] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:56:18.702580 11775 tablet.cc:2369] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 16:56:18 dist-test-slave-xb9n krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888178, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for HTTP/127.30.148.148@KRBTEST.COM
I20260619 16:56:18.761781 12081 raft_consensus.cc:482] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:56:18.761945 12081 raft_consensus.cc:504] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 37315 } } peers { permanent_uuid: "6d8c96cedcfb4508a68f7e68e175a843" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41187 } } peers { permanent_uuid: "d185216317ed4c70b5754eb824efc7a9" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42939 } }
I20260619 16:56:18.762349 12081 leader_election.cc:288] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d8c96cedcfb4508a68f7e68e175a843 (127.30.148.131:41187), d185216317ed4c70b5754eb824efc7a9 (127.30.148.129:42939)
I20260619 16:56:18.767328 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.762928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:34081 (local address 127.30.148.131:41187)
0619 16:56:18.762971 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:18.762973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:18.762981 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:56:18.763010 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:18.763011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:18.763020 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:18.763089 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:18.763540 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.764293 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.766344 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.766929 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.766978 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:18.766987 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:18.767213 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 16:56:18.767254 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:56:18.767563 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.762482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:41187 (local address 127.30.148.130:34081)
0619 16:56:18.762862 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:18.762874 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:18.762927 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:18.763359 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:18.763360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:18.763362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:18.763455 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:18.763457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.764662 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.764663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:18.766236 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:18.766239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.767064 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.767065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:18.767134 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:18.767137 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:56:18.767432 (+ 295us) client_negotiation.cc:240] Negotiation successful
0619 16:56:18.767480 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":99,"threads_started":1}
I20260619 16:56:18.767935 11996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54853a77ebc24e46b60d2c02569d0de1" candidate_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d8c96cedcfb4508a68f7e68e175a843" is_pre_election: true
I20260619 16:56:18.768114 11996 raft_consensus.cc:2454] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 257b6e53ce2248c0b7c7a6c0a799d1a7 in term 0.
I20260619 16:56:18.768383 11797 leader_election.cc:302] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257b6e53ce2248c0b7c7a6c0a799d1a7, 6d8c96cedcfb4508a68f7e68e175a843; no voters:
I20260619 16:56:18.768524 12081 raft_consensus.cc:2781] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:56:18.768575 12081 raft_consensus.cc:482] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:56:18.768599 12081 raft_consensus.cc:3037] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:18.769523 12081 raft_consensus.cc:504] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 37315 } } peers { permanent_uuid: "6d8c96cedcfb4508a68f7e68e175a843" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41187 } } peers { permanent_uuid: "d185216317ed4c70b5754eb824efc7a9" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42939 } }
I20260619 16:56:18.769654 12081 leader_election.cc:288] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [CANDIDATE]: Term 1 election: Requested vote from peers 6d8c96cedcfb4508a68f7e68e175a843 (127.30.148.131:41187), d185216317ed4c70b5754eb824efc7a9 (127.30.148.129:42939)
I20260619 16:56:18.769830 11996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54853a77ebc24e46b60d2c02569d0de1" candidate_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d8c96cedcfb4508a68f7e68e175a843"
I20260619 16:56:18.769904 11996 raft_consensus.cc:3037] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:18.770777 11996 raft_consensus.cc:2454] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 257b6e53ce2248c0b7c7a6c0a799d1a7 in term 1.
I20260619 16:56:18.770946 11797 leader_election.cc:302] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257b6e53ce2248c0b7c7a6c0a799d1a7, 6d8c96cedcfb4508a68f7e68e175a843; no voters:
I20260619 16:56:18.771029 12081 raft_consensus.cc:2781] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:56:18.771158 12081 raft_consensus.cc:686] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 1 LEADER]: Becoming Leader. State: Replica: 257b6e53ce2248c0b7c7a6c0a799d1a7, State: Running, Role: LEADER
I20260619 16:56:18.771257 12081 consensus_queue.cc:237] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 37315 } } peers { permanent_uuid: "6d8c96cedcfb4508a68f7e68e175a843" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41187 } } peers { permanent_uuid: "d185216317ed4c70b5754eb824efc7a9" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42939 } }
I20260619 16:56:18.772034 11576 catalog_manager.cc:5495] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 257b6e53ce2248c0b7c7a6c0a799d1a7 (127.30.148.130). New cstate: current_term: 1 leader_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 37315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d8c96cedcfb4508a68f7e68e175a843" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d185216317ed4c70b5754eb824efc7a9" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42939 } health_report { overall_health: UNKNOWN } } }
I20260619 16:56:18.772699 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.762792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:59845 (local address 127.30.148.129:42939)
0619 16:56:18.762858 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:18.762860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:18.763211 (+ 351us) server_negotiation.cc:407] Connection header received
0619 16:56:18.763312 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:18.763313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:18.763324 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:18.763354 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:18.763724 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.764407 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.770766 (+ 6359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.771361 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.771423 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:18.771432 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:18.772574 (+ 1142us) server_negotiation.cc:299] Negotiation successful
0619 16:56:18.772618 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 16:56:18.772928 12085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.762713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:42939 (local address 127.30.148.130:59845)
0619 16:56:18.763154 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:18.763164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:18.763236 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:18.763597 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:18.763598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:18.763600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:18.763650 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:18.763651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.764501 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.764502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:18.770672 (+ 6170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:18.770676 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.772427 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.772428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:18.772494 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:18.772497 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:56:18.772802 (+ 305us) client_negotiation.cc:240] Negotiation successful
0619 16:56:18.772845 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":4027,"thread_start_us":74,"threads_started":1}
I20260619 16:56:18.773252 11716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54853a77ebc24e46b60d2c02569d0de1" candidate_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d185216317ed4c70b5754eb824efc7a9" is_pre_election: true
I20260619 16:56:18.773422 11716 raft_consensus.cc:2454] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 257b6e53ce2248c0b7c7a6c0a799d1a7 in term 0.
I20260619 16:56:18.773561 11717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54853a77ebc24e46b60d2c02569d0de1" candidate_uuid: "257b6e53ce2248c0b7c7a6c0a799d1a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d185216317ed4c70b5754eb824efc7a9"
I20260619 16:56:18.773643 11717 raft_consensus.cc:3037] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:18.774453 11717 raft_consensus.cc:2454] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 257b6e53ce2248c0b7c7a6c0a799d1a7 in term 1.
I20260619 16:56:18.781247 12071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.778177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51900 (local address 127.30.148.130:37315)
0619 16:56:18.778212 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:18.778214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:18.778249 (+ 35us) server_negotiation.cc:407] Connection header received
0619 16:56:18.778278 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:18.778279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:18.778295 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:18.778358 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:56:18.778489 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.779154 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.780120 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:18.780460 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:18.780532 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:18.780711 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:56:18.780831 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:56:18.780919 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:18.781162 (+ 243us) server_negotiation.cc:299] Negotiation successful
0619 16:56:18.781182 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:56:18.783217 11717 raft_consensus.cc:1261] T 54853a77ebc24e46b60d2c02569d0de1 P d185216317ed4c70b5754eb824efc7a9 [term 1 FOLLOWER]: Refusing update from remote peer 257b6e53ce2248c0b7c7a6c0a799d1a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:56:18.783568 12081 consensus_queue.cc:1035] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d185216317ed4c70b5754eb824efc7a9" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 42939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:56:18.785543 11996 raft_consensus.cc:1261] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843 [term 1 FOLLOWER]: Refusing update from remote peer 257b6e53ce2248c0b7c7a6c0a799d1a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:56:18.785794 12081 consensus_queue.cc:1035] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d8c96cedcfb4508a68f7e68e175a843" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:56:18.790493 12094 mvcc.cc:205] Tried to move back new op lower bound from 7298613980293492736 to 7298613980247584768. Current Snapshot: MvccSnapshot[applied={T|T < 7298613980293492736}]
I20260619 16:56:18.790583 11576 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:56:18.790674 11576 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:56:18.791344 12092 mvcc.cc:205] Tried to move back new op lower bound from 7298613980293492736 to 7298613980247584768. Current Snapshot: MvccSnapshot[applied={T|T < 7298613980293492736}]
I20260619 16:56:18.793378 11576 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e26a3e95403940b69ac27c8b5295af9f: Sending DeleteTablet for 3 replicas of tablet 54853a77ebc24e46b60d2c02569d0de1
I20260619 16:56:18.793942 11824 tablet_service.cc:1514] Processing DeleteTablet for tablet 54853a77ebc24e46b60d2c02569d0de1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:18 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:51884
I20260619 16:56:18.794158 11976 tablet_service.cc:1514] Processing DeleteTablet for tablet 54853a77ebc24e46b60d2c02569d0de1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:18 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:44544
I20260619 16:56:18.794324 11708 tablet_service.cc:1514] Processing DeleteTablet for tablet 54853a77ebc24e46b60d2c02569d0de1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:18 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:58460
I20260619 16:56:18.794780 12100 tablet_replica.cc:330] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7: stopping tablet replica
I20260619 16:56:18.794862 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 11614
I20260619 16:56:18.794907 12101 tablet_replica.cc:330] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843: stopping tablet replica
I20260619 16:56:18.795475 12101 raft_consensus.cc:2229] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:56:18.795595 12100 raft_consensus.cc:2229] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:56:18.795722 12101 raft_consensus.cc:2258] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:56:18.795847 12100 raft_consensus.cc:2258] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:56:18.796396 12100 ts_tablet_manager.cc:1905] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:56:18.796720 12101 ts_tablet_manager.cc:1905] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:56:18.802805 12100 ts_tablet_manager.cc:1918] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:56:18.802971 12101 ts_tablet_manager.cc:1918] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:56:18.803082 12100 log.cc:1196] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/54853a77ebc24e46b60d2c02569d0de1
I20260619 16:56:18.803192 12101 log.cc:1196] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/54853a77ebc24e46b60d2c02569d0de1
I20260619 16:56:18.803542 12100 ts_tablet_manager.cc:1939] T 54853a77ebc24e46b60d2c02569d0de1 P 257b6e53ce2248c0b7c7a6c0a799d1a7: Deleting consensus metadata
I20260619 16:56:18.803860 12101 ts_tablet_manager.cc:1939] T 54853a77ebc24e46b60d2c02569d0de1 P 6d8c96cedcfb4508a68f7e68e175a843: Deleting consensus metadata
I20260619 16:56:18.804323 11560 catalog_manager.cc:4841] TS 257b6e53ce2248c0b7c7a6c0a799d1a7 (127.30.148.130:37315): tablet 54853a77ebc24e46b60d2c02569d0de1 (table test-table [id=d6d77fc9c428445f8e4ac6eb5f907c17]) successfully deleted
I20260619 16:56:18.804844 11562 catalog_manager.cc:4841] TS 6d8c96cedcfb4508a68f7e68e175a843 (127.30.148.131:41187): tablet 54853a77ebc24e46b60d2c02569d0de1 (table test-table [id=d6d77fc9c428445f8e4ac6eb5f907c17]) successfully deleted
W20260619 16:56:18.805938 11562 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:42939 (error 108)
I20260619 16:56:18.812258 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 11778
I20260619 16:56:18.816427 12072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:18.816272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:42939 (local address 127.0.0.1:58472)
0619 16:56:18.816335 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:18.816392 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:42939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260619 16:56:18.816570 11562 catalog_manager.cc:4584] TS d185216317ed4c70b5754eb824efc7a9 (127.30.148.129:42939): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 54853a77ebc24e46b60d2c02569d0de1: Network error: Client connection negotiation failed: client connection to 127.30.148.129:42939: connect: Connection refused (error 111)
I20260619 16:56:18.822443 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 11911
I20260619 16:56:18.828965 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 11543
I20260619 16:56:18.833541 31314 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 16:56:19.167158 31314 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 16:56:19.167256 31314 mini_ranger.cc:66] Stopping Ranger...
I20260619 16:56:19.512579 31314 mini_ranger.cc:68] Stopped Ranger
2026-06-19 16:56:19.512 UTC [10609] LOG: received smart shutdown request
2026-06-19 16:56:19.516 UTC [10609] LOG: background worker "logical replication launcher" (PID 10618) exited with exit code 1
2026-06-19 16:56:19.516 UTC [10613] LOG: shutting down
2026-06-19 16:56:19.653 UTC [10609] LOG: database system is shut down
2026-06-19T16:56:19Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (44686 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:56:19 dist-test-slave-xb9n krb5kdc[12112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:56:19 dist-test-slave-xb9n krb5kdc[12112](info): set up 2 sockets
Jun 19 16:56:19 dist-test-slave-xb9n krb5kdc[12112](info): commencing operation
krb5kdc: starting...
W20260619 16:56:20.793200 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:56:20 dist-test-slave-xb9n krb5kdc[12112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:56:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:56:20.902385 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:44601
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:44601,127.30.148.189:41745,127.30.148.188:44041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:20.980731 12125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:20.980916 12125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:20.980937 12125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:20.982015 12125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:56:20.982048 12125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:20.982061 12125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:20.982074 12125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:56:20.982085 12125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:56:20.983386 12125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:44601,127.30.148.189:41745,127.30.148.188:44041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:44601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:20.983558 12125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:20.983711 12125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:20.988242 12131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:20.988368 12125 server_base.cc:1034] running on GCE node
W20260619 16:56:20.988225 12133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:20.988229 12130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:20.988713 12125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:20.989029 12125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:20.990180 12125 hybrid_clock.cc:648] HybridClock initialized: now 1781888180990175 us; error 34 us; skew 500 ppm
Jun 19 16:56:20 dist-test-slave-xb9n krb5kdc[12112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888180, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:20.996558 12125 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:56:20.997018 12125 webserver.cc:460] Webserver started at http://127.30.148.190:36635/ using document root <none> and password file <none>
I20260619 16:56:20.997241 12125 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:20.997303 12125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:20.997460 12125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:56:20.998445 12125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "f6569adad45147b8893443bb74a3e8ce"
format_stamp: "Formatted at 2026-06-19 16:56:20 on dist-test-slave-xb9n"
server_key: "828aee7716058f88d64c959ab72a5480"
server_key_iv: "9c3f882e78bac8f10510c224a178c17b"
server_key_version: "encryptionkey@0"
I20260619 16:56:20.998793 12125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "f6569adad45147b8893443bb74a3e8ce"
format_stamp: "Formatted at 2026-06-19 16:56:20 on dist-test-slave-xb9n"
server_key: "828aee7716058f88d64c959ab72a5480"
server_key_iv: "9c3f882e78bac8f10510c224a178c17b"
server_key_version: "encryptionkey@0"
I20260619 16:56:21.000041 12125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:56:21.000703 12140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:21.000864 12125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:56:21.000955 12125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "f6569adad45147b8893443bb74a3e8ce"
format_stamp: "Formatted at 2026-06-19 16:56:20 on dist-test-slave-xb9n"
server_key: "828aee7716058f88d64c959ab72a5480"
server_key_iv: "9c3f882e78bac8f10510c224a178c17b"
server_key_version: "encryptionkey@0"
I20260619 16:56:21.001039 12125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:21.008291 12125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:21.010419 12125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:21.010574 12125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:21.014496 12125 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:44601
I20260619 16:56:21.014521 12192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:44601 every 8 connection(s)
I20260619 16:56:21.014834 12125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:56:21.015596 12193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:21.016453 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 12125
I20260619 16:56:21.016567 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:56:21.016757 31314 external_mini_cluster.cc:1428] Setting key a8a0c45d3c2fa5a2fc66bfb09d007eaa
I20260619 16:56:21.017176 12193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888180, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.148.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.189@KRBTEST.COM" created.
I20260619 16:56:21.027619 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.017879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:44601 (local address 127.0.0.1:56120)
0619 16:56:21.018120 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.018162 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:21.018814 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:21.019104 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:21.019106 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:21.019388 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:21.019529 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.019533 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.021013 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.021014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.021784 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.021786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.022519 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.022520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.022573 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.023047 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:21.023055 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:21.024715 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:21.026341 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.026347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.026349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.026544 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.026746 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.026747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.026747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.026821 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.027168 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:21.027174 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:21.027250 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 16:56:21.027345 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 16:56:21.027464 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":82,"threads_started":1}
I20260619 16:56:21.027743 12197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.017879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.30.148.190:44601)
0619 16:56:21.018062 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.018071 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.018248 (+ 177us) server_negotiation.cc:407] Connection header received
0619 16:56:21.018962 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.018964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.018982 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.019050 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.019624 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.020929 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.021877 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.022417 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.022558 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.024800 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.024807 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.024808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.024813 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.026200 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.026613 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.026615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.026615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.026691 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.026866 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.026867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.026867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.027103 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.027165 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.027401 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.027547 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":130,"spinlock_wait_cycles":7424,"thread_start_us":85,"threads_started":1}
I20260619 16:56:21.028442 12193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:56:21.028762 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.028673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:41745 (local address 127.0.0.1:44634)
0619 16:56:21.028706 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.028746 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260619 16:56:21.028913 12143 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.148.189:41745: connect: Connection refused (error 111)
I20260619 16:56:21.029403 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.029282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:41745 (local address 127.0.0.1:44650)
0619 16:56:21.029329 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.029353 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 16:56:21.029512 12193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.189:41745: Network error: Client connection negotiation failed: client connection to 127.30.148.189:41745: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.189.keytab.
Entry for principal kudu/127.30.148.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.189.keytab.
I20260619 16:56:21.041772 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.189:41745
--webserver_interface=127.30.148.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.189.keytab
--principal=kudu/127.30.148.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:44601,127.30.148.189:41745,127.30.148.188:44041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:56:21.078699 12193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } attempt: 1
I20260619 16:56:21.079134 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.079011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:41745 (local address 127.0.0.1:44654)
0619 16:56:21.079069 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.079107 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 16:56:21.079455 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.079369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:41745 (local address 127.0.0.1:44660)
0619 16:56:21.079410 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.079433 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 16:56:21.079535 12193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.189:41745: Network error: Client connection negotiation failed: client connection to 127.30.148.189:41745: connect: Connection refused (error 111)
W20260619 16:56:21.119839 12200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:21.120044 12200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:21.120081 12200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:21.121191 12200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:56:21.121253 12200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:21.121276 12200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:21.121304 12200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:56:21.121331 12200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:56:21.122771 12200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:44601,127.30.148.189:41745,127.30.148.188:44041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.189.keytab
--principal=kudu/127.30.148.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.189:41745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.148.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:21.123029 12200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:21.123238 12200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:21.127795 12208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:21.127792 12205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:21.127821 12200 server_base.cc:1034] running on GCE node
W20260619 16:56:21.127779 12206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:21.128300 12200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:21.128525 12200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:21.129660 12200 hybrid_clock.cc:648] HybridClock initialized: now 1781888181129648 us; error 28 us; skew 500 ppm
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:21.131887 12200 init.cc:377] Logged in from keytab as kudu/127.30.148.189@KRBTEST.COM (short username kudu)
I20260619 16:56:21.132257 12200 webserver.cc:460] Webserver started at http://127.30.148.189:40645/ using document root <none> and password file <none>
I20260619 16:56:21.132465 12200 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:21.132525 12200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:21.132658 12200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:56:21.133657 12200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/data/instance:
uuid: "b8b0b5e49dad4ad6b1322571f6bd5698"
format_stamp: "Formatted at 2026-06-19 16:56:21 on dist-test-slave-xb9n"
server_key: "177292b6209bea2e64f7e52327496540"
server_key_iv: "52098a47f5c9f0b2969fafcea98b46ba"
server_key_version: "encryptionkey@0"
I20260619 16:56:21.134004 12200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/wal/instance:
uuid: "b8b0b5e49dad4ad6b1322571f6bd5698"
format_stamp: "Formatted at 2026-06-19 16:56:21 on dist-test-slave-xb9n"
server_key: "177292b6209bea2e64f7e52327496540"
server_key_iv: "52098a47f5c9f0b2969fafcea98b46ba"
server_key_version: "encryptionkey@0"
I20260619 16:56:21.135197 12200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:56:21.135840 12215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:21.135993 12200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:56:21.136080 12200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/wal
uuid: "b8b0b5e49dad4ad6b1322571f6bd5698"
format_stamp: "Formatted at 2026-06-19 16:56:21 on dist-test-slave-xb9n"
server_key: "177292b6209bea2e64f7e52327496540"
server_key_iv: "52098a47f5c9f0b2969fafcea98b46ba"
server_key_version: "encryptionkey@0"
I20260619 16:56:21.136149 12200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:21.147661 12193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } attempt: 2
I20260619 16:56:21.148097 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.147982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:41745 (local address 127.0.0.1:44676)
0619 16:56:21.148034 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.148069 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 16:56:21.148416 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.148341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:41745 (local address 127.0.0.1:44682)
0619 16:56:21.148377 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.148400 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 16:56:21.148530 12193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.189:41745: Network error: Client connection negotiation failed: client connection to 127.30.148.189:41745: connect: Connection refused (error 111)
I20260619 16:56:21.149993 12200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:21.151618 12200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:21.151757 12200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:21.155438 12200 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.189:41745
I20260619 16:56:21.155504 12267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.189:41745 every 8 connection(s)
I20260619 16:56:21.155791 12200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
I20260619 16:56:21.156574 12268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:21.158042 12268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.189@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:21.165733 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 12200
I20260619 16:56:21.165827 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-1/wal/instance
I20260619 16:56:21.166050 31314 external_mini_cluster.cc:1428] Setting key 3d58b89c0ab1c0044eddcf090d634f6a
I20260619 16:56:21.166152 12197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.158645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56134 (local address 127.30.148.190:44601)
0619 16:56:21.158715 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.158717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.158868 (+ 151us) server_negotiation.cc:407] Connection header received
0619 16:56:21.159287 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.159288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.159300 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.159343 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.159885 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.160345 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.161236 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.161483 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.161534 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.163467 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.163468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.163468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.163474 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.164846 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.165230 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.165231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.165231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.165278 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.165500 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.165501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.165501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.165596 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.165647 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.165984 (+ 337us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.166044 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 16:56:21.166466 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.158609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:44601 (local address 127.0.0.1:56134)
0619 16:56:21.158763 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.158795 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:21.159201 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:21.159420 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:21.159423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:21.159645 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:21.159819 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.159822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.160417 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.160417 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.161171 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.161173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.161549 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.161549 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.161650 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.162107 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:21.162115 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:21.163381 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:21.164922 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.164928 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.164930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.165172 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.165321 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.165321 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.165321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.165449 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.165658 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:21.165665 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:21.165832 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 16:56:21.165970 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 16:56:21.166154 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260619 16:56:21.167030 12268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.189@KRBTEST.COM for kudu/127.30.148.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.148.188@KRBTEST.COM; defaulting to no policy
I20260619 16:56:21.175395 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.167266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:41745 (local address 127.0.0.1:44686)
0619 16:56:21.167323 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.167334 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:21.167374 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:21.168549 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:21.168550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:21.168705 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:21.168770 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.168772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.169693 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.169694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.170560 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.170564 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.170996 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.170997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.171082 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.171427 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:21.171432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:21.172639 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:21.174391 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.174392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.174393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.174557 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.174765 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.174766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.174766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.174819 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.175089 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:21.175089 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:21.175126 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 16:56:21.175194 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 16:56:21.175285 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 16:56:21.175473 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.167317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44686 (local address 127.30.148.189:41745)
0619 16:56:21.168413 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.168417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.168433 (+ 16us) server_negotiation.cc:407] Connection header received
0619 16:56:21.168451 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.168452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.168463 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.168526 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.168853 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.169597 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.170644 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.170920 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.170985 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.172728 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.172740 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.172740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.172746 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.174284 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.174647 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.174649 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.174650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.174702 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.174884 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.174884 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.174885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.175015 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.175097 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.175223 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.175336 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":1072,"thread_start_us":82,"threads_started":1}
I20260619 16:56:21.176015 12268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.188@KRBTEST.COM" created.
I20260619 16:56:21.176390 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.176262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48190)
0619 16:56:21.176309 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.176358 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 16:56:21.176528 12218 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
I20260619 16:56:21.176939 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.176850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48194)
0619 16:56:21.176891 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.176915 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 16:56:21.177034 12268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:44041: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.188.keytab.
Entry for principal kudu/127.30.148.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.188.keytab.
I20260619 16:56:21.190135 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.188:44041
--webserver_interface=127.30.148.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.188.keytab
--principal=kudu/127.30.148.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:44601,127.30.148.189:41745,127.30.148.188:44041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:56:21.226240 12268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } attempt: 1
I20260619 16:56:21.226735 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.226614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48206)
0619 16:56:21.226674 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.226711 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 16:56:21.227075 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.226997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48214)
0619 16:56:21.227036 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.227060 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 16:56:21.227180 12268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:44041: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
I20260619 16:56:21.239686 12193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } attempt: 3
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888180, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.189@KRBTEST.COM
I20260619 16:56:21.246558 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.240057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44700 (local address 127.30.148.189:41745)
0619 16:56:21.240114 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.240116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.240128 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:56:21.240198 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.240198 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.240210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.240255 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.240568 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.241069 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.241805 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.242018 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.242100 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.244142 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.244143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.244143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.244149 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.245524 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.245890 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.245891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.245891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.245940 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.246135 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.246136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.246136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.246229 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.246277 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.246440 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.246500 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 16:56:21.246616 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.240006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:41745 (local address 127.0.0.1:44700)
0619 16:56:21.240059 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.240069 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:21.240134 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:21.240270 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:21.240271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:21.240427 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:21.240490 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.240492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.241150 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.241152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.241711 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.241713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.242100 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.242101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.242180 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.242655 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:21.242659 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:21.244065 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:21.245598 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.245599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.245600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.245818 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.246008 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.246009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.246009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.246076 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.246289 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:21.246290 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:21.246345 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 16:56:21.246418 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 16:56:21.246519 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 16:56:21.247014 12193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:56:21.247342 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.247225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48220)
0619 16:56:21.247280 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.247316 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 16:56:21.247648 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.247561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48232)
0619 16:56:21.247602 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.247629 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 16:56:21.247751 12193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:44041: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
W20260619 16:56:21.272171 12275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:21.272356 12275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:21.272378 12275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:21.273458 12275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:56:21.273492 12275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:21.273506 12275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:21.273519 12275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:56:21.273531 12275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:56:21.274930 12275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:44601,127.30.148.189:41745,127.30.148.188:44041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.188.keytab
--principal=kudu/127.30.148.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.188:44041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.148.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:21.275102 12275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:21.275254 12275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:56:21.278880 12193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } attempt: 1
I20260619 16:56:21.279340 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.279187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48244)
0619 16:56:21.279240 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.279298 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 16:56:21.279749 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.279654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48250)
0619 16:56:21.279689 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.279720 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260619 16:56:21.279758 12283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:21.279808 12281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:21.279886 12193 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:44041: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
W20260619 16:56:21.279984 12280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:21.280215 12275 server_base.cc:1034] running on GCE node
I20260619 16:56:21.280457 12275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:21.280730 12275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:21.281890 12275 hybrid_clock.cc:648] HybridClock initialized: now 1781888181281885 us; error 42 us; skew 500 ppm
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:21.284171 12275 init.cc:377] Logged in from keytab as kudu/127.30.148.188@KRBTEST.COM (short username kudu)
I20260619 16:56:21.284567 12275 webserver.cc:460] Webserver started at http://127.30.148.188:36733/ using document root <none> and password file <none>
I20260619 16:56:21.284780 12275 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:21.284845 12275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:21.284991 12275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:56:21.286006 12275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/data/instance:
uuid: "80371d832b9544b0ac9f4d8f452a1235"
format_stamp: "Formatted at 2026-06-19 16:56:21 on dist-test-slave-xb9n"
server_key: "5a79bf20fd10f807477caeebfb128195"
server_key_iv: "3a77d1b24385637ace9647855fc2ac57"
server_key_version: "encryptionkey@0"
I20260619 16:56:21.286350 12275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/wal/instance:
uuid: "80371d832b9544b0ac9f4d8f452a1235"
format_stamp: "Formatted at 2026-06-19 16:56:21 on dist-test-slave-xb9n"
server_key: "5a79bf20fd10f807477caeebfb128195"
server_key_iv: "3a77d1b24385637ace9647855fc2ac57"
server_key_version: "encryptionkey@0"
I20260619 16:56:21.287532 12275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:56:21.288234 12290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:21.288420 12275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:56:21.288503 12275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/wal
uuid: "80371d832b9544b0ac9f4d8f452a1235"
format_stamp: "Formatted at 2026-06-19 16:56:21 on dist-test-slave-xb9n"
server_key: "5a79bf20fd10f807477caeebfb128195"
server_key_iv: "3a77d1b24385637ace9647855fc2ac57"
server_key_version: "encryptionkey@0"
I20260619 16:56:21.288576 12275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:21.295382 12268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } attempt: 2
I20260619 16:56:21.295729 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.295621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48266)
0619 16:56:21.295676 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.295707 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 16:56:21.296025 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.295932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48282)
0619 16:56:21.295972 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.296009 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 16:56:21.296104 12268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:44041: Network error: Client connection negotiation failed: client connection to 127.30.148.188:44041: connect: Connection refused (error 111)
I20260619 16:56:21.302223 12275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:21.304045 12275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:21.304185 12275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:21.308465 12275 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.188:44041
I20260619 16:56:21.308490 12342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.188:44041 every 8 connection(s)
I20260619 16:56:21.308974 12275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
I20260619 16:56:21.309854 12343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:21.311276 12343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:56:21.314435 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 12275
I20260619 16:56:21.314520 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888036930978-31314-0/minicluster-data/master-2/wal/instance
I20260619 16:56:21.314718 31314 external_mini_cluster.cc:1428] Setting key 7053950ad73ad22d6d5684c1d138abbf
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.188@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.188@KRBTEST.COM
I20260619 16:56:21.321412 12197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.311933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56142 (local address 127.30.148.190:44601)
0619 16:56:21.311997 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.311999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.312181 (+ 182us) server_negotiation.cc:407] Connection header received
0619 16:56:21.312625 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.312627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.312639 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.312719 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.313230 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.313769 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.315105 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.315372 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.315459 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.317972 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.317974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.317974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.317981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.319666 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.320225 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.320226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.320226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.320275 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.320559 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.320559 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.320559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.320757 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.320825 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.321172 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.321262 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 16:56:21.321498 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.311893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:44601 (local address 127.0.0.1:56142)
0619 16:56:21.312056 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.312096 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:21.312521 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:21.312763 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:21.312766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:21.312983 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:21.313157 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.313161 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.313846 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.313847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.314950 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.314954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.315478 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.315479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.315609 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.316098 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:21.316106 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:21.317855 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:21.319824 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.319830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.319832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.320144 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.320365 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.320366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.320366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.320492 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.320847 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:21.320854 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:21.321007 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 16:56:21.321120 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 16:56:21.321271 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260619 16:56:21.322036 12343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:56:21.324558 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.315117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48292 (local address 127.30.148.188:44041)
0619 16:56:21.315283 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.315288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.315333 (+ 45us) server_negotiation.cc:407] Connection header received
0619 16:56:21.315442 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.315447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.315468 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.315554 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.316333 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.317443 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.318222 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.318509 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.318621 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.321069 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.321076 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.321076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.321082 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.323342 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.323685 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.323688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.323689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.323763 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.323972 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.323973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.323973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.324093 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.324203 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.324339 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.324420 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.188@KRBTEST.COM for kudu/127.30.148.189@KRBTEST.COM
I20260619 16:56:21.329315 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.322322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:41745 (local address 127.0.0.1:44714)
0619 16:56:21.322376 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.322387 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:21.322431 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:21.322591 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:21.322592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:21.322722 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:21.322748 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.322750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.323808 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.323809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.324550 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.324553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.324973 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.324974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.325016 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.325420 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:21.325424 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:21.326667 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:21.328327 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.328329 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.328329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.328545 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.328786 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.328787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.328787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.328840 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.329125 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:21.329125 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:21.329154 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 16:56:21.329199 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 16:56:21.329262 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":7}
I20260619 16:56:21.329445 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.322419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.30.148.189:41745)
0619 16:56:21.322481 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.322483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.322493 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:56:21.322519 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.322520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.322530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.322585 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.322828 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.323694 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.324607 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.324896 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.324980 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.326760 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.326761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.326761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.326768 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.328208 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.328634 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.328635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.328635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.328698 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.328908 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.328909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.328909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.329054 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.329139 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.329230 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.329310 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 16:56:21.329797 12343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.188@KRBTEST.COM for kudu/127.30.148.188@KRBTEST.COM
I20260619 16:56:21.333693 12197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.325422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56148 (local address 127.30.148.190:44601)
0619 16:56:21.325469 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.325471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.325479 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:56:21.325500 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.325501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.325510 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.325570 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.326029 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.326435 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.327493 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.328006 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.328073 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.330767 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.330768 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.330768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.330773 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.332150 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.332500 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.332501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.332501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.332555 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.332750 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.332751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.332752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.332856 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.333046 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.333582 (+ 536us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.333638 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 16:56:21.336261 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.330027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48294)
0619 16:56:21.330345 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.330355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:21.330399 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:21.330520 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:21.330520 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:21.330620 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:21.330645 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.330645 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.331270 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.331271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.331732 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.331733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.332090 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.332091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.332136 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.332547 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:21.332552 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:21.333695 (+ 1143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:21.335353 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.335354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.335354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.335555 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.335768 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.335769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.335769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.335821 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.336064 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:21.336064 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:21.336096 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 16:56:21.336142 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 16:56:21.336209 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302}
I20260619 16:56:21.336360 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.330097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48294 (local address 127.30.148.188:44041)
0619 16:56:21.330136 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.330138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.330392 (+ 254us) server_negotiation.cc:407] Connection header received
0619 16:56:21.330448 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.330449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.330460 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.330518 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.330712 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.331190 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.331813 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.332019 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.332093 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.333780 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.333780 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.333781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.333787 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.335281 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.335625 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.335626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.335626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.335699 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.335880 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.335881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.335881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.336000 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.336066 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.336172 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.336278 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":20}
I20260619 16:56:21.338070 12343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235: Bootstrap starting.
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.189@KRBTEST.COM
I20260619 16:56:21.338955 12343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:21.339282 12343 log.cc:824] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:21.340150 12343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235: No bootstrap required, opened a new log
I20260619 16:56:21.341944 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.334308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.30.148.189:41745)
0619 16:56:21.334339 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.334341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.334349 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:56:21.334418 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.334418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.334425 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.334467 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.335063 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.335611 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.336521 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.336714 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.336790 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.339178 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.339179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.339180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.339186 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.340821 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.341154 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.341155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.341156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.341214 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.341448 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.341448 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.341449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.341568 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.341738 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.341776 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.341848 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 16:56:21.342350 12343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } }
I20260619 16:56:21.342489 12343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:21.342533 12343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80371d832b9544b0ac9f4d8f452a1235, State: Initialized, Role: FOLLOWER
I20260619 16:56:21.342653 12343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } }
I20260619 16:56:21.342924 12349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } } }
I20260619 16:56:21.342967 12343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:56:21.343058 12349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:56:21.343688 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260619 16:56:21.344789 12360 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:56:21.344843 12360 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:56:21.345222 12275 master_runner.cc:186] Error getting master registration for 127.30.148.190:44601: OK, instance_id { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" instance_seqno: 1781888181013352 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 16:56:21.347790 12275 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 16:56:21.352357 12197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.345229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56158 (local address 127.30.148.190:44601)
0619 16:56:21.345275 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.345277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.345415 (+ 138us) server_negotiation.cc:407] Connection header received
0619 16:56:21.345510 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.345511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.345522 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.345552 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.346280 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.346956 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.347943 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.348126 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.348183 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.349569 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.349570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.349571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.349577 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.351157 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.351549 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.351550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.351550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.351601 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.351798 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.351799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.351799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.351915 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.351981 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.352185 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.352268 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 16:56:21.355017 12193 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } attempt: 2
I20260619 16:56:21.356688 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.345448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.30.148.189:41745)
0619 16:56:21.345482 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.345484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.349567 (+ 4083us) server_negotiation.cc:407] Connection header received
0619 16:56:21.349638 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.349639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.349650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.349703 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.350356 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.351051 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.351990 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.352272 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.352308 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.353825 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.353826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.353827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.353833 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.355416 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.355931 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.355932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.355932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.355987 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.356204 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.356205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.356205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.356313 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.356406 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.356530 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.356598 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 16:56:21.358026 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.345699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48302 (local address 127.30.148.188:44041)
0619 16:56:21.345737 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.345739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.349725 (+ 3986us) server_negotiation.cc:407] Connection header received
0619 16:56:21.350129 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.350130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.350140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.350170 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.351018 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.351697 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.352932 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.353249 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.353296 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.354854 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.354855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.354855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.354862 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.356625 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.357113 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.357114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.357114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.357176 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.357407 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.357408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.357408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.357569 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.357650 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.357843 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.357918 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888180, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.188@KRBTEST.COM
I20260619 16:56:21.364497 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.355252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48306)
0619 16:56:21.355370 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.355380 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:21.355418 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:21.357631 (+ 2213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:21.357632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:21.357781 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:21.357816 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.357817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.358851 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.358851 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.359322 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.359323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.360039 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.360039 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.360100 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.360545 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:21.360549 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:21.361795 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:21.363640 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.363641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.363641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.363804 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.364001 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.364001 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.364001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.364048 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.364299 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:21.364299 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:21.364332 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 16:56:21.364380 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 16:56:21.364441 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260619 16:56:21.364591 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.355508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48306 (local address 127.30.148.188:44041)
0619 16:56:21.357503 (+ 1995us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.357504 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.357511 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:56:21.357527 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.357528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.357537 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.357611 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.357911 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.358760 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.359405 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.359956 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.360238 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.361872 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.361873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.361873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.361880 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.363525 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.363875 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.363876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.363877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.363934 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.364107 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.364108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.364108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.364227 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.364302 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.364404 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.364476 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1953,"thread_start_us":87,"threads_started":1}
I20260619 16:56:21.365756 12193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce: Bootstrap starting.
I20260619 16:56:21.366344 12193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:21.366758 12193 log.cc:824] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:21.367625 12193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce: No bootstrap required, opened a new log
I20260619 16:56:21.369153 12193 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } }
I20260619 16:56:21.369287 12193 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:21.369335 12193 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6569adad45147b8893443bb74a3e8ce, State: Initialized, Role: FOLLOWER
I20260619 16:56:21.369522 12193 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } }
I20260619 16:56:21.369833 12193 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:56:21.369810 12371 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } } }
I20260619 16:56:21.369894 12371 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [sys.catalog]: This master's current role is: FOLLOWER
W20260619 16:56:21.371500 12382 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:56:21.371610 12382 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:56:21.372605 12125 master_runner.cc:186] Error getting master registration for 127.30.148.189:41745: OK, instance_id { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" instance_seqno: 1781888181154442 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 16:56:21.375366 12125 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 16:56:21.387243 12268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } attempt: 3
Jun 19 16:56:21 dist-test-slave-xb9n krb5kdc[12112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.189@KRBTEST.COM for kudu/127.30.148.188@KRBTEST.COM
I20260619 16:56:21.395291 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.387478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:44041 (local address 127.0.0.1:48308)
0619 16:56:21.387520 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:21.387531 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:21.387564 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:21.387705 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:21.387705 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:21.387903 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:21.387950 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.387953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.388888 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.388889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.389702 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:21.389704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.390183 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.390184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:21.390242 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.390736 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:21.390741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:21.392182 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:21.394047 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.394048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.394049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.394261 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.394471 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:21.394472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:21.394472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:21.394538 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:21.394996 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:21.394998 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:21.395042 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 16:56:21.395100 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 16:56:21.395197 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 16:56:21.395623 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:21.387534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48308 (local address 127.30.148.188:44041)
0619 16:56:21.387569 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:21.387571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:21.387604 (+ 33us) server_negotiation.cc:407] Connection header received
0619 16:56:21.387624 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:21.387624 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:21.387633 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:21.387685 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:21.388035 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.388803 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.389787 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:21.390101 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:21.390176 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:21.392264 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:21.392265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:21.392266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:21.392272 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:21.393965 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.394342 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.394343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.394344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.394399 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:21.394607 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:21.394608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:21.394608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:21.394729 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:21.394802 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:21.395458 (+ 656us) server_negotiation.cc:299] Negotiation successful
0619 16:56:21.395533 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:56:21.396977 12268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698: Bootstrap starting.
I20260619 16:56:21.397708 12268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:21.398048 12268 log.cc:824] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:21.398885 12268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698: No bootstrap required, opened a new log
I20260619 16:56:21.400857 12268 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } }
I20260619 16:56:21.400990 12268 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:21.401022 12268 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8b0b5e49dad4ad6b1322571f6bd5698, State: Initialized, Role: FOLLOWER
I20260619 16:56:21.401152 12268 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } }
I20260619 16:56:21.401499 12268 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 16:56:21.402284 12221 tablet.cc:2369] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:56:21.401465 12385 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } } }
W20260619 16:56:21.403882 12396 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:56:21.403966 12396 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:56:21.403373 12385 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:56:21.405892 12200 master_runner.cc:263] No leader master found from master b8b0b5e49dad4ad6b1322571f6bd5698
I20260619 16:56:21.405933 12200 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 16:56:21.481230 12349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:56:21.481376 12349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } }
I20260619 16:56:21.481612 12349 leader_election.cc:288] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6569adad45147b8893443bb74a3e8ce (127.30.148.190:44601), b8b0b5e49dad4ad6b1322571f6bd5698 (127.30.148.189:41745)
I20260619 16:56:21.482071 12242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80371d832b9544b0ac9f4d8f452a1235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" is_pre_election: true
I20260619 16:56:21.482218 12242 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80371d832b9544b0ac9f4d8f452a1235 in term 0.
I20260619 16:56:21.482610 12293 leader_election.cc:302] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80371d832b9544b0ac9f4d8f452a1235, b8b0b5e49dad4ad6b1322571f6bd5698; no voters:
I20260619 16:56:21.482894 12167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80371d832b9544b0ac9f4d8f452a1235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6569adad45147b8893443bb74a3e8ce" is_pre_election: true
I20260619 16:56:21.483032 12167 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80371d832b9544b0ac9f4d8f452a1235 in term 0.
I20260619 16:56:21.483328 12349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:56:21.483381 12349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:56:21.483403 12349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:21.484701 12349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } }
I20260619 16:56:21.484824 12349 leader_election.cc:288] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [CANDIDATE]: Term 1 election: Requested vote from peers f6569adad45147b8893443bb74a3e8ce (127.30.148.190:44601), b8b0b5e49dad4ad6b1322571f6bd5698 (127.30.148.189:41745)
I20260619 16:56:21.485129 12242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80371d832b9544b0ac9f4d8f452a1235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698"
I20260619 16:56:21.485195 12242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:21.485929 12242 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80371d832b9544b0ac9f4d8f452a1235 in term 1.
I20260619 16:56:21.486120 12293 leader_election.cc:302] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80371d832b9544b0ac9f4d8f452a1235, b8b0b5e49dad4ad6b1322571f6bd5698; no voters:
I20260619 16:56:21.486228 12167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80371d832b9544b0ac9f4d8f452a1235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6569adad45147b8893443bb74a3e8ce"
I20260619 16:56:21.486294 12167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:21.487074 12167 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80371d832b9544b0ac9f4d8f452a1235 in term 1.
I20260619 16:56:21.487974 12349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:56:21.488127 12349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [term 1 LEADER]: Becoming Leader. State: Replica: 80371d832b9544b0ac9f4d8f452a1235, State: Running, Role: LEADER
I20260619 16:56:21.488201 12349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } }
I20260619 16:56:21.488515 12349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80371d832b9544b0ac9f4d8f452a1235. Latest consensus state: current_term: 1 leader_uuid: "80371d832b9544b0ac9f4d8f452a1235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } } }
I20260619 16:56:21.488575 12349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:21.488858 12400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:56:21.489079 12400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:56:21.490875 12242 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [term 1 FOLLOWER]: Refusing update from remote peer 80371d832b9544b0ac9f4d8f452a1235: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:56:21.491211 12398 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:56:21.492391 12385 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80371d832b9544b0ac9f4d8f452a1235. Latest consensus state: current_term: 1 leader_uuid: "80371d832b9544b0ac9f4d8f452a1235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } } }
I20260619 16:56:21.492466 12385 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:56:21.492909 12167 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [term 1 FOLLOWER]: Refusing update from remote peer 80371d832b9544b0ac9f4d8f452a1235: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:56:21.493167 12398 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:56:21.494225 12371 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80371d832b9544b0ac9f4d8f452a1235. Latest consensus state: current_term: 1 leader_uuid: "80371d832b9544b0ac9f4d8f452a1235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } } }
I20260619 16:56:21.494323 12371 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:56:21.495301 12398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80371d832b9544b0ac9f4d8f452a1235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } } }
I20260619 16:56:21.495378 12398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:21.495968 12371 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80371d832b9544b0ac9f4d8f452a1235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } } }
I20260619 16:56:21.496043 12371 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:56:21.496187 12401 mvcc.cc:205] Tried to move back new op lower bound from 7298613991384674304 to 7298613991376179200. Current Snapshot: MvccSnapshot[applied={T|T < 7298613991384674304}]
I20260619 16:56:21.496305 12404 mvcc.cc:205] Tried to move back new op lower bound from 7298613991384674304 to 7298613991376179200. Current Snapshot: MvccSnapshot[applied={T|T < 7298613991384674304}]
I20260619 16:56:21.497159 12400 catalog_manager.cc:1347] Generated new cluster ID: 97dc4a06c1ad45a6a83b27ff486259b7
I20260619 16:56:21.497200 12400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:56:21.501412 12349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80371d832b9544b0ac9f4d8f452a1235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } } }
I20260619 16:56:21.501487 12349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235 [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:21.501870 12385 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80371d832b9544b0ac9f4d8f452a1235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6569adad45147b8893443bb74a3e8ce" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44601 } } peers { permanent_uuid: "b8b0b5e49dad4ad6b1322571f6bd5698" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 41745 } } peers { permanent_uuid: "80371d832b9544b0ac9f4d8f452a1235" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 44041 } } }
I20260619 16:56:21.501935 12385 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:56:21.510727 12400 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:56:21.511363 12400 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:56:21.523536 12400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80371d832b9544b0ac9f4d8f452a1235: Generated new TSK 0
I20260619 16:56:21.523696 12400 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:56:22.372059 12382 catalog_manager.cc:1259] Loaded cluster ID: 97dc4a06c1ad45a6a83b27ff486259b7
I20260619 16:56:22.372131 12382 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce: loading cluster ID for follower catalog manager: success
I20260619 16:56:22.372967 12382 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce: acquiring CA information for follower catalog manager: success
I20260619 16:56:22.373306 12382 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f6569adad45147b8893443bb74a3e8ce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 16:56:22.404409 12396 catalog_manager.cc:1259] Loaded cluster ID: 97dc4a06c1ad45a6a83b27ff486259b7
I20260619 16:56:22.404474 12396 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698: loading cluster ID for follower catalog manager: success
I20260619 16:56:22.405304 12396 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698: acquiring CA information for follower catalog manager: success
I20260619 16:56:22.405655 12396 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b8b0b5e49dad4ad6b1322571f6bd5698: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 16:56:22.569852 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 12125
I20260619 16:56:22.574958 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 12200
I20260619 16:56:22.579604 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 12275
2026-06-19T16:56:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2824 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:56:22 dist-test-slave-xb9n krb5kdc[12416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:56:22 dist-test-slave-xb9n krb5kdc[12416](info): set up 2 sockets
Jun 19 16:56:22 dist-test-slave-xb9n krb5kdc[12416](info): commencing operation
krb5kdc: starting...
W20260619 16:56:23.605280 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:56:23 dist-test-slave-xb9n krb5kdc[12416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:56:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:56:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:56:23.713851 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:44581
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:40003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:44581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:23.791272 12429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:23.791448 12429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:23.791469 12429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:23.792467 12429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:56:23.792500 12429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:23.792512 12429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:23.792524 12429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:56:23.792536 12429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:56:23.793889 12429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:40003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:44581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:44581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:23.794054 12429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:23.794205 12429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:56:23.798846 12429 server_base.cc:1034] running on GCE node
W20260619 16:56:23.798775 12434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:23.798789 12435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:23.798794 12437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:23.799275 12429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:23.799513 12429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:23.800650 12429 hybrid_clock.cc:648] HybridClock initialized: now 1781888183800641 us; error 28 us; skew 500 ppm
Jun 19 16:56:23 dist-test-slave-xb9n krb5kdc[12416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888183, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:23.803011 12429 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:56:23.803449 12429 webserver.cc:460] Webserver started at http://127.30.148.190:36129/ using document root <none> and password file <none>
I20260619 16:56:23.803659 12429 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:23.803723 12429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:23.803874 12429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:56:23.804845 12429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "b705768f965f428faf13dd99610c0b75"
format_stamp: "Formatted at 2026-06-19 16:56:23 on dist-test-slave-xb9n"
server_key: "5533bb778e2279aa6062711cf269768e"
server_key_iv: "e869737d391ffba10da6fb4fd73a8275"
server_key_version: "encryptionkey@0"
I20260619 16:56:23.805191 12429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "b705768f965f428faf13dd99610c0b75"
format_stamp: "Formatted at 2026-06-19 16:56:23 on dist-test-slave-xb9n"
server_key: "5533bb778e2279aa6062711cf269768e"
server_key_iv: "e869737d391ffba10da6fb4fd73a8275"
server_key_version: "encryptionkey@0"
I20260619 16:56:23.806528 12429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:56:23.807209 12444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:23.807397 12429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:56:23.807493 12429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "b705768f965f428faf13dd99610c0b75"
format_stamp: "Formatted at 2026-06-19 16:56:23 on dist-test-slave-xb9n"
server_key: "5533bb778e2279aa6062711cf269768e"
server_key_iv: "e869737d391ffba10da6fb4fd73a8275"
server_key_version: "encryptionkey@0"
I20260619 16:56:23.807576 12429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:23.830042 12429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:23.831718 12429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:23.831881 12429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:23.835682 12429 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:44581
I20260619 16:56:23.835714 12496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:44581 every 8 connection(s)
I20260619 16:56:23.836035 12429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:56:23.836798 12497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:23.838268 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 12429
I20260619 16:56:23.838392 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:56:23.838574 31314 external_mini_cluster.cc:1428] Setting key 7f19915da40853804a485b36d8435ca4
I20260619 16:56:23.839300 12497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75: Bootstrap starting.
I20260619 16:56:23.839880 12497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:23.840206 12497 log.cc:824] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:23.840863 12497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75: No bootstrap required, opened a new log
I20260619 16:56:23.843092 12497 raft_consensus.cc:348] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b705768f965f428faf13dd99610c0b75" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44581 } }
I20260619 16:56:23.843281 12497 raft_consensus.cc:374] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:23.843330 12497 raft_consensus.cc:729] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b705768f965f428faf13dd99610c0b75, State: Initialized, Role: FOLLOWER
I20260619 16:56:23.843456 12497 consensus_queue.cc:260] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b705768f965f428faf13dd99610c0b75" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44581 } }
I20260619 16:56:23.843550 12497 raft_consensus.cc:388] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:56:23.843587 12497 raft_consensus.cc:482] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:56:23.843684 12497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:23.844178 12497 raft_consensus.cc:504] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b705768f965f428faf13dd99610c0b75" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44581 } }
I20260619 16:56:23.844296 12497 leader_election.cc:302] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b705768f965f428faf13dd99610c0b75; no voters:
I20260619 16:56:23.844503 12497 leader_election.cc:288] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:56:23.844527 12502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:56:23.844633 12502 raft_consensus.cc:686] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [term 1 LEADER]: Becoming Leader. State: Replica: b705768f965f428faf13dd99610c0b75, State: Running, Role: LEADER
I20260619 16:56:23.844708 12502 consensus_queue.cc:237] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b705768f965f428faf13dd99610c0b75" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44581 } }
I20260619 16:56:23.844749 12497 sys_catalog.cc:564] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:56:23.845010 12502 sys_catalog.cc:455] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b705768f965f428faf13dd99610c0b75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b705768f965f428faf13dd99610c0b75" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44581 } } }
I20260619 16:56:23.845142 12502 sys_catalog.cc:458] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [sys.catalog]: This master's current role is: LEADER
Jun 19 16:56:23 dist-test-slave-xb9n krb5kdc[12416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:23.845353 12502 sys_catalog.cc:455] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b705768f965f428faf13dd99610c0b75. Latest consensus state: current_term: 1 leader_uuid: "b705768f965f428faf13dd99610c0b75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b705768f965f428faf13dd99610c0b75" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 44581 } } }
I20260619 16:56:23.845481 12502 sys_catalog.cc:458] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75 [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:23.846019 12508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:56:23.846526 12508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:56:23.848098 12508 catalog_manager.cc:1347] Generated new cluster ID: 274b6851d66d4605aed73e89c321e5cc
I20260619 16:56:23.848155 12508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:56:23.849701 12500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:23.839065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50072 (local address 127.30.148.190:44581)
0619 16:56:23.839235 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:23.839242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:23.839266 (+ 24us) server_negotiation.cc:407] Connection header received
0619 16:56:23.839717 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:23.839724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:23.839765 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:23.839877 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:23.840634 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:23.841569 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:23.842688 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:23.843049 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:23.843165 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:23.845880 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:23.845884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:23.845884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:23.845897 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:23.847804 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:23.848253 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:23.848256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:23.848257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:23.848355 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:23.848537 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:23.848537 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:23.848537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:23.848791 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:23.848905 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:23.849388 (+ 483us) server_negotiation.cc:299] Negotiation successful
0619 16:56:23.849506 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260619 16:56:23.850646 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:56:23.863535 12508 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:56:23.864008 12508 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:56:23.871245 12508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b705768f965f428faf13dd99610c0b75: Generated new TSK 0
I20260619 16:56:23.871433 12508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:56:23.904348 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 12429
2026-06-19T16:56:23Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1323 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:56:23 dist-test-slave-xb9n krb5kdc[12524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:56:23 dist-test-slave-xb9n krb5kdc[12524](info): set up 2 sockets
Jun 19 16:56:23 dist-test-slave-xb9n krb5kdc[12524](info): commencing operation
krb5kdc: starting...
W20260619 16:56:24.929455 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:56:25 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:56:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:56:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:56:25.040815 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33849
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:33849
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:25.119204 12537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:25.119426 12537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:25.119464 12537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:25.120602 12537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:56:25.120666 12537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:25.120690 12537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:25.120720 12537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:56:25.120745 12537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:56:25.122257 12537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:33849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:25.122524 12537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:25.122730 12537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:25.127533 12542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:25.127522 12543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:25.127513 12545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:25.127552 12537 server_base.cc:1034] running on GCE node
I20260619 16:56:25.127933 12537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:25.128242 12537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:25.129424 12537 hybrid_clock.cc:648] HybridClock initialized: now 1781888185129416 us; error 37 us; skew 500 ppm
Jun 19 16:56:25 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888185, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:25.131783 12537 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:56:25.132206 12537 webserver.cc:460] Webserver started at http://127.30.148.190:35757/ using document root <none> and password file <none>
I20260619 16:56:25.132424 12537 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:25.132490 12537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:25.132635 12537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:56:25.133705 12537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "c9c97365988242ebb7523edbabf45a54"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "e0a1620d343539ae93d4f69b790e950f"
server_key_iv: "e9eb3430fb5fa703c1b550383d893d8c"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.134059 12537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "c9c97365988242ebb7523edbabf45a54"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "e0a1620d343539ae93d4f69b790e950f"
server_key_iv: "e9eb3430fb5fa703c1b550383d893d8c"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.135304 12537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:56:25.135977 12552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:25.136138 12537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:56:25.136231 12537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "c9c97365988242ebb7523edbabf45a54"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "e0a1620d343539ae93d4f69b790e950f"
server_key_iv: "e9eb3430fb5fa703c1b550383d893d8c"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.136304 12537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:25.149629 12537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:25.151298 12537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:25.151453 12537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:25.155254 12537 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:33849
I20260619 16:56:25.155283 12604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:33849 every 8 connection(s)
I20260619 16:56:25.155591 12537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:56:25.156500 12605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:25.158608 12605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Bootstrap starting.
I20260619 16:56:25.159111 12605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:25.159337 12605 log.cc:824] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:25.159952 12605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: No bootstrap required, opened a new log
I20260619 16:56:25.161458 12605 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } }
I20260619 16:56:25.161586 12605 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:25.161608 12605 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9c97365988242ebb7523edbabf45a54, State: Initialized, Role: FOLLOWER
I20260619 16:56:25.161700 12605 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } }
I20260619 16:56:25.161755 12605 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:56:25.161777 12605 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:56:25.161803 12605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:25.162536 12605 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } }
I20260619 16:56:25.162618 12605 leader_election.cc:302] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9c97365988242ebb7523edbabf45a54; no voters:
I20260619 16:56:25.162726 12605 leader_election.cc:288] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:56:25.162791 12608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:56:25.162958 12608 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 1 LEADER]: Becoming Leader. State: Replica: c9c97365988242ebb7523edbabf45a54, State: Running, Role: LEADER
I20260619 16:56:25.162987 12605 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:56:25.163048 12608 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } }
I20260619 16:56:25.163405 12609 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9c97365988242ebb7523edbabf45a54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } } }
I20260619 16:56:25.163414 12610 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9c97365988242ebb7523edbabf45a54. Latest consensus state: current_term: 1 leader_uuid: "c9c97365988242ebb7523edbabf45a54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } } }
I20260619 16:56:25.163501 12609 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:25.163532 12610 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:25.164167 12617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:56:25.164778 12617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:56:25.164791 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 12537
I20260619 16:56:25.164891 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:56:25.165099 31314 external_mini_cluster.cc:1428] Setting key ca8b48271e1f1384b9fedcb15324bf25
I20260619 16:56:25.166492 12617 catalog_manager.cc:1347] Generated new cluster ID: 8243259d3b32474fad31d258453face3
I20260619 16:56:25.166548 12617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:56:25 dist-test-slave-xb9n krb5kdc[12524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:25.178287 12627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.165512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55546 (local address 127.30.148.190:33849)
0619 16:56:25.169470 (+ 3958us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:25.169477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:25.169501 (+ 24us) server_negotiation.cc:407] Connection header received
0619 16:56:25.169583 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:25.169587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:25.169625 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:25.169753 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:25.170366 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.171244 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.171990 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.172398 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.172506 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.175001 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:25.175005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:25.175005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:25.175016 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:25.176692 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:25.177090 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:25.177093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:25.177093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:25.177185 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:25.177410 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:25.177411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:25.177411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:25.177656 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:25.177804 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:25.177927 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 16:56:25.178080 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3905,"thread_start_us":100,"threads_started":1}
I20260619 16:56:25.196398 12617 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:56:25.196890 12617 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:56:25.201241 12617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Generated new TSK 0
I20260619 16:56:25.201455 12617 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:56:25.234287 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33849
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:25.322005 12631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:25.322188 12631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:25.322211 12631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:25.323297 12631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:25.323335 12631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:25.323360 12631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:56:25.324743 12631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:25.324972 12631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:25.325196 12631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:25.330053 12639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:25.330164 12631 server_base.cc:1034] running on GCE node
W20260619 16:56:25.330050 12637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:25.330122 12636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:25.330470 12631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:25.330701 12631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:25.331859 12631 hybrid_clock.cc:648] HybridClock initialized: now 1781888185331840 us; error 35 us; skew 500 ppm
Jun 19 16:56:25 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888185, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:25.334311 12631 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:56:25.334712 12631 webserver.cc:460] Webserver started at http://127.30.148.129:45733/ using document root <none> and password file <none>
I20260619 16:56:25.334926 12631 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:25.335000 12631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:25.335150 12631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:56:25.336122 12631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "4ed013ce13f643f7a4e667c4e89fb8c2"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "7532aeb2205bb4694128b46e0e0904f2"
server_key_iv: "8875a8894a4ce085f9789336c63a1fc5"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.336468 12631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "4ed013ce13f643f7a4e667c4e89fb8c2"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "7532aeb2205bb4694128b46e0e0904f2"
server_key_iv: "8875a8894a4ce085f9789336c63a1fc5"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.337836 12631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:56:25.338665 12646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:25.338842 12631 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:56:25.338948 12631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "4ed013ce13f643f7a4e667c4e89fb8c2"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "7532aeb2205bb4694128b46e0e0904f2"
server_key_iv: "8875a8894a4ce085f9789336c63a1fc5"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.339020 12631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:25.367208 12631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:25.368985 12631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:25.369091 12631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:25.369268 12631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:25.369596 12631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:25.369628 12631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:25.369650 12631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:25.369663 12631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:25.375116 12631 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:37441
I20260619 16:56:25.375159 12759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:37441 every 8 connection(s)
I20260619 16:56:25.375500 12631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:56:25.379094 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 12631
I20260619 16:56:25.379207 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:56:25.379393 31314 external_mini_cluster.cc:1428] Setting key 5f1884980a719e436b029e4424232ed8
Jun 19 16:56:25 dist-test-slave-xb9n krb5kdc[12524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888185, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:25.384999 12762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.376187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:55121)
0619 16:56:25.376355 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:25.376389 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:25.376839 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:25.377046 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:25.377049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:25.377644 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:25.377825 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.377829 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.378485 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.378486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.379361 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.379363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.379766 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.379767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.379904 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.380455 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:25.380463 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:25.381854 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:25.383680 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:25.383687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:25.383707 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:25.383966 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:25.384181 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:25.384182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:25.384182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:25.384275 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:25.384564 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:25.384568 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:25.384670 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 16:56:25.384744 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 16:56:25.384853 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260619 16:56:25.385033 12627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.376232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:55121 (local address 127.30.148.190:33849)
0619 16:56:25.376308 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:25.376311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:25.376471 (+ 160us) server_negotiation.cc:407] Connection header received
0619 16:56:25.376928 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:25.376928 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:25.376942 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:25.376995 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:25.377912 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.378408 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.379460 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.379700 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.379755 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.381950 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:25.381951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:25.381952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:25.381960 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:25.383593 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:25.384044 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:25.384045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:25.384046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:25.384110 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:25.384340 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:25.384341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:25.384342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:25.384488 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:25.384563 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:25.384805 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 16:56:25.384898 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 16:56:25.385497 12760 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33849
I20260619 16:56:25.385607 12760 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:25.385774 12760 heartbeater.cc:511] Master 127.30.148.190:33849 requested a full tablet report, sending...
I20260619 16:56:25.386304 12569 ts_manager.cc:194] Registered new tserver with Master: 4ed013ce13f643f7a4e667c4e89fb8c2 (127.30.148.129:37441)
I20260619 16:56:25.387225 12569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:55121
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:56:25.404021 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33849
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:25.481848 12765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:25.482054 12765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:25.482089 12765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:25.483211 12765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:25.483282 12765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:25.483331 12765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:56:25.484712 12765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:25.484962 12765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:25.485172 12765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:25.489926 12771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:25.489897 12773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:25.489890 12770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:25.489969 12765 server_base.cc:1034] running on GCE node
I20260619 16:56:25.490257 12765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:25.490447 12765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:25.491581 12765 hybrid_clock.cc:648] HybridClock initialized: now 1781888185491569 us; error 28 us; skew 500 ppm
Jun 19 16:56:25 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888185, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:25.493999 12765 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:56:25.494387 12765 webserver.cc:460] Webserver started at http://127.30.148.130:41331/ using document root <none> and password file <none>
I20260619 16:56:25.494616 12765 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:25.494680 12765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:25.494824 12765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:56:25.495761 12765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "e262d168e0e9460594babcb1b67a2f12"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "52c14837f25bf91aa62238dfd08708fc"
server_key_iv: "6d56b9051313adbf7e852bb42f213a5a"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.496101 12765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "e262d168e0e9460594babcb1b67a2f12"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "52c14837f25bf91aa62238dfd08708fc"
server_key_iv: "6d56b9051313adbf7e852bb42f213a5a"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.497327 12765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:56:25.498072 12780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:25.498235 12765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:56:25.498327 12765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "e262d168e0e9460594babcb1b67a2f12"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "52c14837f25bf91aa62238dfd08708fc"
server_key_iv: "6d56b9051313adbf7e852bb42f213a5a"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.498417 12765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:25.507863 12765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:25.510084 12765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:25.510239 12765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:25.510453 12765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:25.510737 12765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:25.510787 12765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:25.510824 12765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:25.510854 12765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:25.516341 12765 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:45255
I20260619 16:56:25.516376 12893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:45255 every 8 connection(s)
I20260619 16:56:25.516700 12765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:56:25.518165 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 12765
I20260619 16:56:25.518266 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:56:25.518395 31314 external_mini_cluster.cc:1428] Setting key 78eb621dd871d3308c0812f5faad22d6
Jun 19 16:56:25 dist-test-slave-xb9n krb5kdc[12524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888185, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:25.525894 12627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.517412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:44013 (local address 127.30.148.190:33849)
0619 16:56:25.517483 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:25.517485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:25.517652 (+ 167us) server_negotiation.cc:407] Connection header received
0619 16:56:25.518145 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:25.518147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:25.518159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:25.518211 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:25.519137 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.519640 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.520575 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.520775 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.520861 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.523168 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:25.523170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:25.523170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:25.523178 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:25.524552 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:25.524950 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:25.524951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:25.524952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:25.525017 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:25.525258 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:25.525259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:25.525259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:25.525417 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:25.525478 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:25.525730 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 16:56:25.525809 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 16:56:25.526077 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.517341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:44013)
0619 16:56:25.517532 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:25.517577 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:25.518036 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:25.518274 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:25.518278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:25.518857 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:25.519042 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.519047 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.519737 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.519738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.520488 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.520490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.520869 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.520870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.521057 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.521577 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:25.521587 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:25.523071 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:25.524643 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:25.524649 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:25.524652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:25.524859 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:25.525089 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:25.525089 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:25.525090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:25.525186 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:25.525469 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:25.525475 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:25.525600 (+ 125us) client_negotiation.cc:769] Sending connection context
0619 16:56:25.525698 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 16:56:25.525844 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
I20260619 16:56:25.526458 12894 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33849
I20260619 16:56:25.526574 12894 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:25.526733 12894 heartbeater.cc:511] Master 127.30.148.190:33849 requested a full tablet report, sending...
I20260619 16:56:25.527097 12569 ts_manager.cc:194] Registered new tserver with Master: e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255)
I20260619 16:56:25.527421 12569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:44013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:56:25.543325 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33849
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:25.621094 12899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:25.621263 12899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:25.621285 12899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:25.622443 12899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:25.622493 12899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:25.622519 12899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:56:25.623878 12899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:25.624109 12899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:25.624292 12899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:25.628947 12905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:25.628935 12904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:25.628937 12907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:25.629113 12899 server_base.cc:1034] running on GCE node
I20260619 16:56:25.629308 12899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:25.629609 12899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:25.630743 12899 hybrid_clock.cc:648] HybridClock initialized: now 1781888185630730 us; error 31 us; skew 500 ppm
Jun 19 16:56:25 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888185, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:25.633011 12899 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:56:25.633442 12899 webserver.cc:460] Webserver started at http://127.30.148.131:40259/ using document root <none> and password file <none>
I20260619 16:56:25.633708 12899 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:25.633767 12899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:25.633953 12899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:56:25.634915 12899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "54ab8d08f87946c69ceacb3e0097182b"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "a1af613bebcb59c6a11ad122450afe8e"
server_key_iv: "05ca71285e516d47d2a826f40642107b"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.635241 12899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "54ab8d08f87946c69ceacb3e0097182b"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "a1af613bebcb59c6a11ad122450afe8e"
server_key_iv: "05ca71285e516d47d2a826f40642107b"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.636595 12899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:56:25.637298 12914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:25.637482 12899 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:56:25.637559 12899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "54ab8d08f87946c69ceacb3e0097182b"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "a1af613bebcb59c6a11ad122450afe8e"
server_key_iv: "05ca71285e516d47d2a826f40642107b"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.637609 12899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:25.646670 12899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:25.648486 12899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:25.648583 12899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:25.648729 12899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:25.648947 12899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:25.648976 12899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:25.648995 12899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:25.649008 12899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:25.654453 12899 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:39425
I20260619 16:56:25.654534 13027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:39425 every 8 connection(s)
I20260619 16:56:25.654793 12899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:56:25.657454 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 12899
I20260619 16:56:25.657526 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:56:25.657711 31314 external_mini_cluster.cc:1428] Setting key 8b854b11c1e173ec8b30fb086f20d4a4
Jun 19 16:56:25 dist-test-slave-xb9n krb5kdc[12524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888185, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:25.663878 12627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.655455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:41683 (local address 127.30.148.190:33849)
0619 16:56:25.655501 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:25.655503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:25.655711 (+ 208us) server_negotiation.cc:407] Connection header received
0619 16:56:25.656289 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:25.656290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:25.656302 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:25.656364 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:25.657251 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.657956 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.659138 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.659320 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.659373 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.661211 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:25.661212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:25.661213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:25.661219 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:25.662689 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:25.663072 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:25.663073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:25.663073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:25.663123 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:25.663317 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:25.663318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:25.663318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:25.663409 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:25.663483 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:25.663743 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 16:56:25.663811 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:56:25.664002 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.655428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:41683)
0619 16:56:25.655583 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:25.655622 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:25.656184 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:25.656429 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:25.656432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:25.656968 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:25.657150 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.657154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.658059 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.658061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.659070 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.659073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.659386 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.659387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.659486 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.659893 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:25.659906 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:25.661128 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:25.662764 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:25.662769 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:25.662771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:25.663012 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:25.663163 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:25.663163 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:25.663164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:25.663278 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:25.663527 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:25.663531 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:25.663632 (+ 101us) client_negotiation.cc:769] Sending connection context
0619 16:56:25.663714 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 16:56:25.663819 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260619 16:56:25.664686 13028 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33849
I20260619 16:56:25.664781 13028 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:25.664942 13028 heartbeater.cc:511] Master 127.30.148.190:33849 requested a full tablet report, sending...
I20260619 16:56:25.665318 12569 ts_manager.cc:194] Registered new tserver with Master: 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425)
I20260619 16:56:25.665704 12569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:41683
I20260619 16:56:25.669305 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:56:25.675877 12627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.670655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55552 (local address 127.30.148.190:33849)
0619 16:56:25.670708 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:25.670710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:25.670827 (+ 117us) server_negotiation.cc:407] Connection header received
0619 16:56:25.670863 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:25.670864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:25.670871 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:25.670911 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:25.671421 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.671853 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.672515 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.672675 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.672720 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.673742 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:25.673743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:25.673743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:25.673748 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:25.675037 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:25.675308 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:25.675309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:25.675309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:25.675354 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:25.675487 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:25.675487 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:25.675487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:25.675597 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:25.675674 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:25.675755 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 16:56:25.675803 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 16:56:25.677506 12569 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:56:25.678973 12569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:56:25.684984 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.681098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:39425 (local address 127.0.0.1:40474)
0619 16:56:25.681421 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:25.681430 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:25.681526 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:25.681747 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:25.681749 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:25.681750 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:25.681784 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.681785 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.683071 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.683071 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.683812 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.683814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.684692 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.684693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.684768 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.684781 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 16:56:25.684851 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 16:56:25.684903 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":1,"thread_start_us":37,"threads_started":1}
I20260619 16:56:25.684999 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.681166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.30.148.131:39425)
0619 16:56:25.681304 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:25.681308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:25.681499 (+ 191us) server_negotiation.cc:407] Connection header received
0619 16:56:25.681604 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:25.681607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:25.681621 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:25.681692 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:25.681861 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.682938 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.683903 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.684604 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.684684 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.684735 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:25.684879 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 16:56:25.684918 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260619 16:56:25.685217 13036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.680908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:45255 (local address 127.0.0.1:55612)
0619 16:56:25.681393 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:25.681402 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:25.681452 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:25.681677 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:25.681678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:25.681680 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:25.681727 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.681729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.682620 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.682621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.684005 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.684008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.684968 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.684969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.685049 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.685054 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:56:25.685119 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 16:56:25.685155 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":53,"threads_started":1}
I20260619 16:56:25.685330 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.680901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:37441 (local address 127.0.0.1:36868)
0619 16:56:25.681177 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:25.681216 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:25.681281 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:25.681617 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:25.681620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:25.681622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:25.681702 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.681705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.683058 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.683058 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.684149 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.684151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.684917 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.684918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.684986 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.684989 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:56:25.685194 (+ 205us) client_negotiation.cc:240] Negotiation successful
0619 16:56:25.685240 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":28,"thread_start_us":65,"threads_started":1}
I20260619 16:56:25.685542 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.681064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55612 (local address 127.30.148.130:45255)
0619 16:56:25.681258 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:25.681262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:25.681468 (+ 206us) server_negotiation.cc:407] Connection header received
0619 16:56:25.681517 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:25.681520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:25.681537 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:25.681608 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:25.681804 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.682523 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.684083 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.684789 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.684881 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.684933 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:25.685420 (+ 487us) server_negotiation.cc:299] Negotiation successful
0619 16:56:25.685466 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":83,"threads_started":1}
I20260619 16:56:25.685660 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.680991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36868 (local address 127.30.148.129:37441)
0619 16:56:25.681345 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:25.681349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:25.681359 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:56:25.681432 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:25.681435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:25.681452 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:25.681531 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:25.681783 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.682954 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.684230 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.684758 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.684829 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.684883 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:25.685525 (+ 642us) server_negotiation.cc:299] Negotiation successful
0619 16:56:25.685567 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":82,"threads_started":1}
I20260619 16:56:25.685806 12962 tablet_service.cc:1467] Processing CreateTablet for tablet 699c733110044d41a3f992b8452a79a6 (DEFAULT_TABLE table=test-table [id=fc6527fd343447bab44c0b59242f696c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:56:25.686192 12962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 699c733110044d41a3f992b8452a79a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:25.686365 12828 tablet_service.cc:1467] Processing CreateTablet for tablet 699c733110044d41a3f992b8452a79a6 (DEFAULT_TABLE table=test-table [id=fc6527fd343447bab44c0b59242f696c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:56:25.686668 12828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 699c733110044d41a3f992b8452a79a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:25.689535 13042 tablet_bootstrap.cc:492] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b: Bootstrap starting.
I20260619 16:56:25.689848 13043 tablet_bootstrap.cc:492] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12: Bootstrap starting.
I20260619 16:56:25.690178 13042 tablet_bootstrap.cc:654] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:25.690341 12694 tablet_service.cc:1467] Processing CreateTablet for tablet 699c733110044d41a3f992b8452a79a6 (DEFAULT_TABLE table=test-table [id=fc6527fd343447bab44c0b59242f696c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:56:25.690522 13043 tablet_bootstrap.cc:654] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:25.690659 12694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 699c733110044d41a3f992b8452a79a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:25.690727 13042 log.cc:824] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:25.691061 13043 log.cc:824] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:25.691481 13042 tablet_bootstrap.cc:492] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b: No bootstrap required, opened a new log
I20260619 16:56:25.691550 13042 ts_tablet_manager.cc:1397] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:56:25.691793 13043 tablet_bootstrap.cc:492] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12: No bootstrap required, opened a new log
I20260619 16:56:25.691869 13043 ts_tablet_manager.cc:1397] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:56:25.693017 13042 raft_consensus.cc:348] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } }
I20260619 16:56:25.693157 13042 raft_consensus.cc:374] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:25.693197 13042 raft_consensus.cc:729] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ab8d08f87946c69ceacb3e0097182b, State: Initialized, Role: FOLLOWER
I20260619 16:56:25.693300 13042 consensus_queue.cc:260] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } }
I20260619 16:56:25.693598 13042 ts_tablet_manager.cc:1428] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:56:25.693734 13028 heartbeater.cc:503] Master 127.30.148.190:33849 was elected leader, sending a full tablet report...
I20260619 16:56:25.693856 13043 raft_consensus.cc:348] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } }
I20260619 16:56:25.693945 13047 tablet_bootstrap.cc:492] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Bootstrap starting.
I20260619 16:56:25.694027 13043 raft_consensus.cc:374] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:25.694088 13043 raft_consensus.cc:729] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e262d168e0e9460594babcb1b67a2f12, State: Initialized, Role: FOLLOWER
I20260619 16:56:25.694221 13043 consensus_queue.cc:260] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } }
I20260619 16:56:25.694522 12894 heartbeater.cc:503] Master 127.30.148.190:33849 was elected leader, sending a full tablet report...
I20260619 16:56:25.694607 13047 tablet_bootstrap.cc:654] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:25.694748 13043 ts_tablet_manager.cc:1428] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:56:25.695204 13047 log.cc:824] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:25.695941 13047 tablet_bootstrap.cc:492] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: No bootstrap required, opened a new log
I20260619 16:56:25.696002 13047 ts_tablet_manager.cc:1397] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:56:25.697316 13047 raft_consensus.cc:348] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } }
I20260619 16:56:25.697510 13047 raft_consensus.cc:374] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:25.697566 13047 raft_consensus.cc:729] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed013ce13f643f7a4e667c4e89fb8c2, State: Initialized, Role: FOLLOWER
I20260619 16:56:25.697669 13047 consensus_queue.cc:260] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } }
I20260619 16:56:25.697901 13047 ts_tablet_manager.cc:1428] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:56:25.697926 12760 heartbeater.cc:503] Master 127.30.148.190:33849 was elected leader, sending a full tablet report...
W20260619 16:56:25.767235 12895 tablet.cc:2369] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:56:25.812788 13046 raft_consensus.cc:482] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:56:25.812904 13046 raft_consensus.cc:504] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } }
I20260619 16:56:25.813234 13046 leader_election.cc:288] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed013ce13f643f7a4e667c4e89fb8c2 (127.30.148.129:37441), e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255)
I20260619 16:56:25.817188 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.813524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:34911 (local address 127.30.148.129:37441)
0619 16:56:25.813566 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:25.813568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:25.813745 (+ 177us) server_negotiation.cc:407] Connection header received
0619 16:56:25.813781 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:25.813783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:25.813790 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:25.813837 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:25.813989 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.814675 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.816273 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.816832 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.816916 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.816924 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:25.817083 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 16:56:25.817120 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:56:25.817188 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.813506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:49559 (local address 127.30.148.130:45255)
0619 16:56:25.813553 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:25.813555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:25.813567 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:56:25.813624 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:25.813625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:25.813641 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:25.813691 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:25.813899 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.814657 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.816273 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.816839 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.816908 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.816918 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:25.817083 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 16:56:25.817121 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:56:25.817202 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.813438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:45255 (local address 127.30.148.131:49559)
0619 16:56:25.813492 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:25.813512 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:25.813556 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:25.813720 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:25.813721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:25.813722 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:25.813799 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.813800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.814764 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.814765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.816181 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.816184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.816920 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.816921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.817009 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.817011 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:56:25.817059 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 16:56:25.817113 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":126}
I20260619 16:56:25.817212 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.813451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:37441 (local address 127.30.148.131:34911)
0619 16:56:25.813645 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:25.813653 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:25.813723 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:25.813856 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:25.813857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:25.813859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:25.813927 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.813928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.814764 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.814765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.816181 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:25.816184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.816921 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.816921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:25.816997 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.817002 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:56:25.817060 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 16:56:25.817112 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":83,"thread_start_us":109,"threads_started":1}
I20260619 16:56:25.817525 12848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "699c733110044d41a3f992b8452a79a6" candidate_uuid: "54ab8d08f87946c69ceacb3e0097182b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e262d168e0e9460594babcb1b67a2f12" is_pre_election: true
I20260619 16:56:25.817683 12848 raft_consensus.cc:2454] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ab8d08f87946c69ceacb3e0097182b in term 0.
I20260619 16:56:25.817762 12714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "699c733110044d41a3f992b8452a79a6" candidate_uuid: "54ab8d08f87946c69ceacb3e0097182b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" is_pre_election: true
I20260619 16:56:25.817939 12714 raft_consensus.cc:2454] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ab8d08f87946c69ceacb3e0097182b in term 0.
I20260619 16:56:25.817967 12918 leader_election.cc:302] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ab8d08f87946c69ceacb3e0097182b, e262d168e0e9460594babcb1b67a2f12; no voters:
I20260619 16:56:25.818097 13046 raft_consensus.cc:2781] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:56:25.818151 13046 raft_consensus.cc:482] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:56:25.818178 13046 raft_consensus.cc:3037] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:25.818842 13046 raft_consensus.cc:504] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } }
I20260619 16:56:25.818969 13046 leader_election.cc:288] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [CANDIDATE]: Term 1 election: Requested vote from peers 4ed013ce13f643f7a4e667c4e89fb8c2 (127.30.148.129:37441), e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255)
I20260619 16:56:25.819135 12714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "699c733110044d41a3f992b8452a79a6" candidate_uuid: "54ab8d08f87946c69ceacb3e0097182b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2"
I20260619 16:56:25.819139 12848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "699c733110044d41a3f992b8452a79a6" candidate_uuid: "54ab8d08f87946c69ceacb3e0097182b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e262d168e0e9460594babcb1b67a2f12"
I20260619 16:56:25.819211 12848 raft_consensus.cc:3037] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:25.819231 12714 raft_consensus.cc:3037] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:25.819903 12714 raft_consensus.cc:2454] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ab8d08f87946c69ceacb3e0097182b in term 1.
I20260619 16:56:25.819911 12848 raft_consensus.cc:2454] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ab8d08f87946c69ceacb3e0097182b in term 1.
I20260619 16:56:25.820106 12915 leader_election.cc:302] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed013ce13f643f7a4e667c4e89fb8c2, 54ab8d08f87946c69ceacb3e0097182b; no voters:
I20260619 16:56:25.820206 13046 raft_consensus.cc:2781] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:56:25.820344 13046 raft_consensus.cc:686] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 1 LEADER]: Becoming Leader. State: Replica: 54ab8d08f87946c69ceacb3e0097182b, State: Running, Role: LEADER
I20260619 16:56:25.820437 13046 consensus_queue.cc:237] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } }
I20260619 16:56:25.821238 12569 catalog_manager.cc:5495] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b reported cstate change: term changed from 0 to 1, leader changed from <none> to 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131). New cstate: current_term: 1 leader_uuid: "54ab8d08f87946c69ceacb3e0097182b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } health_report { overall_health: HEALTHY } } }
I20260619 16:56:25.827370 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:25.825342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40482 (local address 127.30.148.131:39425)
0619 16:56:25.825407 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:25.825409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:25.825417 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:56:25.825509 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:25.825509 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:25.825525 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:25.825605 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:56:25.825713 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.826147 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.826810 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:25.826983 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:25.827028 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:25.827148 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:56:25.827225 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:56:25.827280 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:25.827313 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 16:56:25.827328 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 16:56:25.828835 12714 raft_consensus.cc:1261] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Refusing update from remote peer 54ab8d08f87946c69ceacb3e0097182b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:56:25.828831 12848 raft_consensus.cc:1261] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12 [term 1 FOLLOWER]: Refusing update from remote peer 54ab8d08f87946c69ceacb3e0097182b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:56:25.829149 13046 consensus_queue.cc:1035] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:56:25.829294 13052 consensus_queue.cc:1035] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:56:25.834208 12569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:56:25.834291 12569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:56:25.834996 12569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Sending DeleteTablet for 3 replicas of tablet 699c733110044d41a3f992b8452a79a6
I20260619 16:56:25.835299 12694 tablet_service.cc:1514] Processing DeleteTablet for tablet 699c733110044d41a3f992b8452a79a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:25 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:36868
I20260619 16:56:25.835327 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 12631
I20260619 16:56:25.835337 12828 tablet_service.cc:1514] Processing DeleteTablet for tablet 699c733110044d41a3f992b8452a79a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:25 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:55612
I20260619 16:56:25.836023 12962 tablet_service.cc:1514] Processing DeleteTablet for tablet 699c733110044d41a3f992b8452a79a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:25 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:40474
I20260619 16:56:25.836241 13066 tablet_replica.cc:330] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b: stopping tablet replica
I20260619 16:56:25.836262 13065 tablet_replica.cc:330] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12: stopping tablet replica
I20260619 16:56:25.836354 13065 raft_consensus.cc:2229] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:56:25.836362 13066 raft_consensus.cc:2229] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:56:25.836458 13065 raft_consensus.cc:2258] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:56:25.836464 13066 raft_consensus.cc:2258] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:56:25.836866 13066 ts_tablet_manager.cc:1905] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:56:25.836866 13065 ts_tablet_manager.cc:1905] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:56:25.838418 13065 ts_tablet_manager.cc:1918] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:56:25.838423 13066 ts_tablet_manager.cc:1918] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:56:25.838491 13066 log.cc:1196] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/699c733110044d41a3f992b8452a79a6
I20260619 16:56:25.838500 13065 log.cc:1196] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/699c733110044d41a3f992b8452a79a6
I20260619 16:56:25.838819 13065 ts_tablet_manager.cc:1939] T 699c733110044d41a3f992b8452a79a6 P e262d168e0e9460594babcb1b67a2f12: Deleting consensus metadata
I20260619 16:56:25.838829 13066 ts_tablet_manager.cc:1939] T 699c733110044d41a3f992b8452a79a6 P 54ab8d08f87946c69ceacb3e0097182b: Deleting consensus metadata
I20260619 16:56:25.839190 12556 catalog_manager.cc:4841] TS e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255): tablet 699c733110044d41a3f992b8452a79a6 (table test-table [id=fc6527fd343447bab44c0b59242f696c]) successfully deleted
I20260619 16:56:25.839308 12556 catalog_manager.cc:4841] TS 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425): tablet 699c733110044d41a3f992b8452a79a6 (table test-table [id=fc6527fd343447bab44c0b59242f696c]) successfully deleted
W20260619 16:56:25.840538 12553 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:37441 (error 108)
W20260619 16:56:25.841576 13037 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:56:25.840864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:37441 (local address 127.0.0.1:36870)
0619 16:56:25.840916 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:25.840925 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:25.840958 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:25.841528 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:37441: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 16:56:25.841688 12553 catalog_manager.cc:4584] TS 4ed013ce13f643f7a4e667c4e89fb8c2 (127.30.148.129:37441): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 699c733110044d41a3f992b8452a79a6: Network error: Client connection negotiation failed: client connection to 127.30.148.129:37441: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 16:56:25.841794 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 12765
I20260619 16:56:25.846522 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 12899
I20260619 16:56:25.850812 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 12537
I20260619 16:56:25.856240 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33849
--webserver_interface=127.30.148.190
--webserver_port=35757
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:33849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:25.934474 13068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:25.934664 13068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:25.934684 13068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:25.935811 13068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:56:25.935849 13068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:25.935863 13068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:25.935875 13068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:56:25.935912 13068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:56:25.937312 13068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:33849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=35757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:25.937589 13068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:25.937815 13068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:25.946036 13074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:25.946036 13077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:25.946026 13075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:25.946269 13068 server_base.cc:1034] running on GCE node
I20260619 16:56:25.946535 13068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:25.946828 13068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:25.947963 13068 hybrid_clock.cc:648] HybridClock initialized: now 1781888185947952 us; error 25 us; skew 500 ppm
Jun 19 16:56:25 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888185, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:25.950461 13068 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:56:25.950910 13068 webserver.cc:460] Webserver started at http://127.30.148.190:35757/ using document root <none> and password file <none>
I20260619 16:56:25.951119 13068 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:25.951169 13068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:25.952417 13068 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:25.953028 13084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:25.953192 13068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:56:25.953294 13068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "c9c97365988242ebb7523edbabf45a54"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "e0a1620d343539ae93d4f69b790e950f"
server_key_iv: "e9eb3430fb5fa703c1b550383d893d8c"
server_key_version: "encryptionkey@0"
I20260619 16:56:25.953719 13068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:25.968969 13068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:25.971362 13068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:25.971520 13068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:25.975431 13068 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:33849
I20260619 16:56:25.975462 13136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:33849 every 8 connection(s)
I20260619 16:56:25.975831 13068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:56:25.977293 13137 sys_catalog.cc:263] Verifying existing consensus state
I20260619 16:56:25.977943 13137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Bootstrap starting.
I20260619 16:56:25.980786 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 13068
I20260619 16:56:25.981119 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:37441
--local_ip_for_outbound_sockets=127.30.148.129
--tserver_master_addrs=127.30.148.190:33849
--webserver_port=45733
--webserver_interface=127.30.148.129
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:56:25.986467 13137 log.cc:824] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:25.988253 13137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:56:25.988559 13137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Bootstrap complete.
I20260619 16:56:25.990787 13137 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } }
I20260619 16:56:25.990998 13137 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9c97365988242ebb7523edbabf45a54, State: Initialized, Role: FOLLOWER
I20260619 16:56:25.991158 13137 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } }
I20260619 16:56:25.991212 13137 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:56:25.991278 13137 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:56:25.991340 13137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 1 FOLLOWER]: Advancing to term 2
I20260619 16:56:25.992282 13137 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } }
I20260619 16:56:25.992370 13137 leader_election.cc:302] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9c97365988242ebb7523edbabf45a54; no voters:
I20260619 16:56:25.992543 13137 leader_election.cc:288] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 16:56:25.992574 13141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 16:56:25.992698 13141 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 2 LEADER]: Becoming Leader. State: Replica: c9c97365988242ebb7523edbabf45a54, State: Running, Role: LEADER
I20260619 16:56:25.992815 13137 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:56:25.992856 13141 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } }
I20260619 16:56:25.993148 13141 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9c97365988242ebb7523edbabf45a54. Latest consensus state: current_term: 2 leader_uuid: "c9c97365988242ebb7523edbabf45a54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } } }
I20260619 16:56:25.993244 13141 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:25.993152 13142 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c9c97365988242ebb7523edbabf45a54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } } }
I20260619 16:56:25.993418 13142 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:25.993467 13148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:56:25.994270 13148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:56:25.994376 13148 catalog_manager.cc:1259] Loaded cluster ID: 8243259d3b32474fad31d258453face3
I20260619 16:56:25.994414 13148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:56:25.994704 13148 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 16:56:25.999413 13148 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 16:56:26.001720 13148 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:56:26.001919 13148 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Loaded TSK: 0
I20260619 16:56:26.002147 13148 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 16:56:26.063067 13139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:26.063277 13139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:26.063315 13139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:26.064419 13139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:26.064486 13139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:26.064530 13139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:56:26.065986 13139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:37441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=45733
--tserver_master_addrs=127.30.148.190:33849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:26.066217 13139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:26.066426 13139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:56:26.072090 13139 server_base.cc:1034] running on GCE node
W20260619 16:56:26.072116 13165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:26.072203 13164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:26.072381 13167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:26.072583 13139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:26.072834 13139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:26.073977 13139 hybrid_clock.cc:648] HybridClock initialized: now 1781888186073967 us; error 27 us; skew 500 ppm
Jun 19 16:56:26 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888186, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:26.080333 13139 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:56:26.080797 13139 webserver.cc:460] Webserver started at http://127.30.148.129:45733/ using document root <none> and password file <none>
I20260619 16:56:26.081015 13139 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:26.081071 13139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:26.082470 13139 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:26.083148 13174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:26.083323 13139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:26.083416 13139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "4ed013ce13f643f7a4e667c4e89fb8c2"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "7532aeb2205bb4694128b46e0e0904f2"
server_key_iv: "8875a8894a4ce085f9789336c63a1fc5"
server_key_version: "encryptionkey@0"
I20260619 16:56:26.083741 13139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:26.096305 13139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:26.098418 13139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:26.098567 13139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:26.098773 13139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:26.099160 13181 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 16:56:26.100944 13139 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 16:56:26.100999 13139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 16:56:26.101052 13139 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 16:56:26.101717 13139 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 16:56:26.101771 13139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 16:56:26.101842 13181 tablet_bootstrap.cc:492] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Bootstrap starting.
I20260619 16:56:26.107678 13139 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:37441
I20260619 16:56:26.108072 13288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:37441 every 8 connection(s)
I20260619 16:56:26.108119 13139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:56:26.112668 13181 log.cc:824] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Log is configured to *not* fsync() on all Append() calls
Jun 19 16:56:26 dist-test-slave-xb9n krb5kdc[12524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888186, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:26.115204 13181 tablet_bootstrap.cc:492] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:56:26.115520 13181 tablet_bootstrap.cc:492] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Bootstrap complete.
I20260619 16:56:26.116219 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 13139
I20260619 16:56:26.116497 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:45255
--local_ip_for_outbound_sockets=127.30.148.130
--tserver_master_addrs=127.30.148.190:33849
--webserver_port=41331
--webserver_interface=127.30.148.130
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:56:26.118280 13181 ts_tablet_manager.cc:1397] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.010s
I20260619 16:56:26.119695 13291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.108958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:37943)
0619 16:56:26.109146 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:26.109192 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:26.109750 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:26.110167 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:26.110170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:26.110450 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:26.110615 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.110619 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.111743 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.111744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.112511 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.112513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.113289 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.113290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.113454 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.113995 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:26.114002 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:26.115383 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:26.117584 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:26.117590 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:26.117592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:26.117909 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:26.118202 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:26.118203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:26.118203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:26.118323 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:26.119099 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:26.119104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:26.119245 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 16:56:26.119332 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 16:56:26.119500 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260619 16:56:26.119637 13181 raft_consensus.cc:348] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } }
I20260619 16:56:26.119968 13181 raft_consensus.cc:729] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed013ce13f643f7a4e667c4e89fb8c2, State: Initialized, Role: FOLLOWER
I20260619 16:56:26.120092 13181 consensus_queue.cc:260] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } }
I20260619 16:56:26.120342 13181 ts_tablet_manager.cc:1428] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260619 16:56:26.120739 13292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.109053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:37943 (local address 127.30.148.190:33849)
0619 16:56:26.109489 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:26.109498 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:26.109537 (+ 39us) server_negotiation.cc:407] Connection header received
0619 16:56:26.109968 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:26.109972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:26.110012 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:26.110138 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:26.110727 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.111614 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.112606 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.113201 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.113302 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.115505 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:26.115509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:26.115510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:26.115521 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:26.117491 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:26.118019 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:26.118022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:26.118022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:26.118133 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:26.118466 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:26.118467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:26.118467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:26.118740 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:26.120232 (+ 1492us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:26.120311 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 16:56:26.120588 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":80,"threads_started":1}
I20260619 16:56:26.121537 13289 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33849
I20260619 16:56:26.121627 13289 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:26.121794 13289 heartbeater.cc:511] Master 127.30.148.190:33849 requested a full tablet report, sending...
I20260619 16:56:26.122428 13101 ts_manager.cc:194] Registered new tserver with Master: 4ed013ce13f643f7a4e667c4e89fb8c2 (127.30.148.129:37441)
I20260619 16:56:26.123548 13101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:37943
I20260619 16:56:26.124337 13289 heartbeater.cc:503] Master 127.30.148.190:33849 was elected leader, sending a full tablet report...
I20260619 16:56:26.128450 13297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.123222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36882 (local address 127.30.148.129:37441)
0619 16:56:26.123389 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:26.123393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:26.125558 (+ 2165us) server_negotiation.cc:407] Connection header received
0619 16:56:26.125648 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:26.125651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:26.125667 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:26.125735 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:26.125932 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.126623 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.127594 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.128049 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.128138 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.128187 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:26.128348 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 16:56:26.128383 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
I20260619 16:56:26.128450 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.122990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:37441 (local address 127.0.0.1:36882)
0619 16:56:26.125467 (+ 2477us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:26.125494 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:26.125576 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:26.125793 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:26.125795 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:26.125797 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:26.125856 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.125858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.126709 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.126710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.127493 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.127494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.128136 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.128137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.128216 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.128227 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 16:56:26.128331 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 16:56:26.128383 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2447,"thread_start_us":80,"threads_started":1}
I20260619 16:56:26.129062 13223 tablet_service.cc:1514] Processing DeleteTablet for tablet 699c733110044d41a3f992b8452a79a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:25 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:36882
I20260619 16:56:26.129539 13298 tablet_replica.cc:330] stopping tablet replica
I20260619 16:56:26.129655 13298 raft_consensus.cc:2229] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:56:26.129719 13298 raft_consensus.cc:2258] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:56:26.129993 13298 ts_tablet_manager.cc:1905] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:56:26.131919 13298 ts_tablet_manager.cc:1918] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:56:26.132004 13298 log.cc:1196] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal/wals/699c733110044d41a3f992b8452a79a6
I20260619 16:56:26.132227 13298 ts_tablet_manager.cc:1939] T 699c733110044d41a3f992b8452a79a6 P 4ed013ce13f643f7a4e667c4e89fb8c2: Deleting consensus metadata
I20260619 16:56:26.132685 13085 catalog_manager.cc:4841] TS 4ed013ce13f643f7a4e667c4e89fb8c2 (127.30.148.129:37441): tablet 699c733110044d41a3f992b8452a79a6 (table test-table [id=fc6527fd343447bab44c0b59242f696c]) successfully deleted
W20260619 16:56:26.200507 13294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:26.200683 13294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:26.200704 13294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:26.201828 13294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:26.201866 13294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:26.201891 13294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:56:26.203308 13294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:45255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=41331
--tserver_master_addrs=127.30.148.190:33849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:26.203471 13294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:26.203691 13294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:26.208391 13304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:26.208443 13303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:26.208401 13306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:26.208590 13294 server_base.cc:1034] running on GCE node
I20260619 16:56:26.208751 13294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:26.208945 13294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:26.210100 13294 hybrid_clock.cc:648] HybridClock initialized: now 1781888186210088 us; error 25 us; skew 500 ppm
Jun 19 16:56:26 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888186, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:26.213147 13294 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:56:26.213654 13294 webserver.cc:460] Webserver started at http://127.30.148.130:41331/ using document root <none> and password file <none>
I20260619 16:56:26.213840 13294 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:26.213904 13294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:26.215050 13294 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:26.215529 13313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:26.215691 13294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:26.215776 13294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "e262d168e0e9460594babcb1b67a2f12"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "52c14837f25bf91aa62238dfd08708fc"
server_key_iv: "6d56b9051313adbf7e852bb42f213a5a"
server_key_version: "encryptionkey@0"
I20260619 16:56:26.216053 13294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:26.256121 13294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:26.258136 13294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:26.258237 13294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:26.258450 13294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:26.258751 13294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:26.258792 13294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:26.258814 13294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:26.258862 13294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:26.264396 13294 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:45255
I20260619 16:56:26.264474 13426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:45255 every 8 connection(s)
I20260619 16:56:26.264798 13294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:56:26 dist-test-slave-xb9n krb5kdc[12524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888186, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:26.272960 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 13294
I20260619 16:56:26.273234 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:39425
--local_ip_for_outbound_sockets=127.30.148.131
--tserver_master_addrs=127.30.148.190:33849
--webserver_port=40259
--webserver_interface=127.30.148.131
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:56:26.274048 13429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.265545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:56309)
0619 16:56:26.265719 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:26.265754 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:26.266249 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:26.266503 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:26.266507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:26.267060 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:26.267241 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.267246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.267975 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.267976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.268778 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.268780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.269184 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.269185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.269321 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.269847 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:26.269855 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:26.271264 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:26.272628 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:26.272634 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:26.272636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:26.272872 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:26.273123 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:26.273124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:26.273124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:26.273233 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:26.273561 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:26.273567 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:26.273683 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 16:56:26.273772 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 16:56:26.273886 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260619 16:56:26.274046 13292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.265534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:56309 (local address 127.30.148.190:33849)
0619 16:56:26.265608 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:26.265610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:26.265838 (+ 228us) server_negotiation.cc:407] Connection header received
0619 16:56:26.266372 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:26.266373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:26.266386 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:26.266438 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:26.267355 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.267879 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.268886 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.269092 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.269174 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.271348 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:26.271350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:26.271350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:26.271357 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:26.272550 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:26.272947 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:26.272948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:26.272949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:26.273014 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:26.273306 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:26.273307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:26.273308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:26.273484 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:26.273556 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:26.273815 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 16:56:26.273915 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 16:56:26.274561 13427 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33849
I20260619 16:56:26.274677 13427 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:26.274850 13427 heartbeater.cc:511] Master 127.30.148.190:33849 requested a full tablet report, sending...
I20260619 16:56:26.275264 13101 ts_manager.cc:194] Registered new tserver with Master: e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255)
I20260619 16:56:26.275640 13101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:56309
W20260619 16:56:26.353233 13430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:26.353487 13430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:26.353515 13430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:26.354676 13430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:26.354715 13430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:26.354769 13430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:56:26.356353 13430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:39425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=40259
--tserver_master_addrs=127.30.148.190:33849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:26.356592 13430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:26.356828 13430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:26.361677 13436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:26.361735 13435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:26.361699 13438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:26.361936 13430 server_base.cc:1034] running on GCE node
I20260619 16:56:26.362157 13430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:26.362394 13430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:26.363557 13430 hybrid_clock.cc:648] HybridClock initialized: now 1781888186363550 us; error 34 us; skew 500 ppm
Jun 19 16:56:26 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888186, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:26.365978 13430 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:56:26.366410 13430 webserver.cc:460] Webserver started at http://127.30.148.131:40259/ using document root <none> and password file <none>
I20260619 16:56:26.366595 13430 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:26.366644 13430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:26.367827 13430 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:26.368355 13445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:26.368535 13430 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:26.368619 13430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "54ab8d08f87946c69ceacb3e0097182b"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "a1af613bebcb59c6a11ad122450afe8e"
server_key_iv: "05ca71285e516d47d2a826f40642107b"
server_key_version: "encryptionkey@0"
I20260619 16:56:26.368922 13430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:26.384398 13430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:26.386571 13430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:26.386721 13430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:26.386967 13430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:26.387290 13430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:26.387341 13430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:26.387377 13430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:26.387404 13430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:26.393028 13430 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:39425
I20260619 16:56:26.393061 13558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:39425 every 8 connection(s)
I20260619 16:56:26.393416 13430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:56:26.398483 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 13430
Jun 19 16:56:26 dist-test-slave-xb9n krb5kdc[12524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888186, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:26.402608 13292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.394147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:58645 (local address 127.30.148.190:33849)
0619 16:56:26.394208 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:26.394210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:26.394447 (+ 237us) server_negotiation.cc:407] Connection header received
0619 16:56:26.394898 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:26.394899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:26.394908 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:26.394957 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:26.395812 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.396301 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.397232 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.397440 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.397515 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.399680 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:26.399681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:26.399682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:26.399689 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:26.401276 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:26.401676 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:26.401677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:26.401678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:26.401735 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:26.401991 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:26.401992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:26.401992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:26.402102 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:26.402170 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:26.402446 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 16:56:26.402523 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 16:56:26.402771 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.394146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:58645)
0619 16:56:26.394306 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:26.394344 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:26.394820 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:26.395018 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:26.395021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:26.395533 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:26.395725 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.395728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.396378 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.396379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.397151 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.397153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.397523 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.397524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.397636 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.398090 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:26.398097 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:26.399594 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:26.401333 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:26.401339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:26.401341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:26.401596 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:26.401813 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:26.401813 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:26.401814 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:26.401916 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:26.402175 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:26.402181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:26.402311 (+ 130us) client_negotiation.cc:769] Sending connection context
0619 16:56:26.402410 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 16:56:26.402553 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260619 16:56:26.403256 13559 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33849
I20260619 16:56:26.403350 13559 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:26.403519 13559 heartbeater.cc:511] Master 127.30.148.190:33849 requested a full tablet report, sending...
I20260619 16:56:26.403945 13101 ts_manager.cc:194] Registered new tserver with Master: 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425)
I20260619 16:56:26.404446 13101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:58645
I20260619 16:56:26.405726 13563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.398873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55562 (local address 127.30.148.190:33849)
0619 16:56:26.399019 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:26.399022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:26.399033 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:56:26.399088 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:26.399088 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:26.399099 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:26.399160 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:26.399742 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.400199 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.401253 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.401484 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.401579 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.402782 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:26.402783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:26.402784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:26.402791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:26.404477 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:26.404843 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:26.404844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:26.404844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:26.404904 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:26.405210 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:26.405211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:26.405211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:26.405314 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:26.405515 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:26.405552 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 16:56:26.405628 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260619 16:56:26.406422 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:56:26.408833 13563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.406732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55574 (local address 127.30.148.190:33849)
0619 16:56:26.406766 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:26.406768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:26.406915 (+ 147us) server_negotiation.cc:407] Connection header received
0619 16:56:26.406947 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:26.406948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:26.406972 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:26.407024 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:56:26.407090 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.407500 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.408174 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.408363 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.408417 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.408542 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:56:26.408623 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:56:26.408679 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:26.408712 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 16:56:26.408730 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:56:26.409112 13101 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:56:26.410691 13101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:56:26.412606 13223 tablet_service.cc:1467] Processing CreateTablet for tablet 585c5661f24a47d4a638fcc43c3b6f0c (DEFAULT_TABLE table=test-table [id=510ad7d48ff44ccd8aad0cc72ab07344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:56:26.412799 13223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585c5661f24a47d4a638fcc43c3b6f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:26.414160 13181 tablet_bootstrap.cc:492] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: Bootstrap starting.
I20260619 16:56:26.414464 13181 tablet_bootstrap.cc:654] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:26.414983 13181 tablet_bootstrap.cc:492] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: No bootstrap required, opened a new log
I20260619 16:56:26.415061 13181 ts_tablet_manager.cc:1397] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 16:56:26.415169 13181 raft_consensus.cc:348] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:26.415236 13181 raft_consensus.cc:374] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:26.415282 13181 raft_consensus.cc:729] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed013ce13f643f7a4e667c4e89fb8c2, State: Initialized, Role: FOLLOWER
I20260619 16:56:26.415333 13181 consensus_queue.cc:260] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:26.415414 13181 ts_tablet_manager.cc:1428] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:26.415967 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.412370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:39425 (local address 127.0.0.1:40496)
0619 16:56:26.412415 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:26.412425 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:26.412457 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:26.412848 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:26.412849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:26.412851 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:26.412884 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.412885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.413907 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.413908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.414813 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.414815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.415729 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.415730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.415801 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.415804 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:56:26.415862 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 16:56:26.415906 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 16:56:26.416275 13566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.412531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40496 (local address 127.30.148.131:39425)
0619 16:56:26.412682 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:26.412686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:26.412696 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:56:26.412733 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:26.412734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:26.412745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:26.412835 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:26.412943 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.413794 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.414896 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.415638 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.416094 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.416156 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:26.416181 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 16:56:26.416214 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260619 16:56:26.416846 13493 tablet_service.cc:1467] Processing CreateTablet for tablet 585c5661f24a47d4a638fcc43c3b6f0c (DEFAULT_TABLE table=test-table [id=510ad7d48ff44ccd8aad0cc72ab07344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:56:26.417114 13493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585c5661f24a47d4a638fcc43c3b6f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:26.417840 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.412479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:45255 (local address 127.0.0.1:55620)
0619 16:56:26.412697 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:26.412707 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:26.412752 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:26.414646 (+ 1894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:26.414647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:26.414648 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:26.414682 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.414683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.415953 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.415954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.416846 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.416848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.417623 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.417624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.417685 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.417688 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:56:26.417741 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 16:56:26.417778 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":78,"threads_started":1}
I20260619 16:56:26.417891 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.412575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55620 (local address 127.30.148.130:45255)
0619 16:56:26.414372 (+ 1797us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:26.414377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:26.414388 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:56:26.414426 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:26.414428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:26.414445 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:26.414642 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:26.414755 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.415840 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.416927 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.417535 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.417613 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.417662 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:26.417757 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 16:56:26.417795 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1758,"thread_start_us":75,"threads_started":1}
I20260619 16:56:26.418519 13361 tablet_service.cc:1467] Processing CreateTablet for tablet 585c5661f24a47d4a638fcc43c3b6f0c (DEFAULT_TABLE table=test-table [id=510ad7d48ff44ccd8aad0cc72ab07344]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:56:26.418761 13361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585c5661f24a47d4a638fcc43c3b6f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:26.420089 13569 tablet_bootstrap.cc:492] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b: Bootstrap starting.
I20260619 16:56:26.420719 13569 tablet_bootstrap.cc:654] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:26.421275 13569 log.cc:824] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:26.422020 13571 tablet_bootstrap.cc:492] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12: Bootstrap starting.
I20260619 16:56:26.422075 13569 tablet_bootstrap.cc:492] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b: No bootstrap required, opened a new log
I20260619 16:56:26.422137 13569 ts_tablet_manager.cc:1397] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 16:56:26.422626 13571 tablet_bootstrap.cc:654] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:26.423121 13571 log.cc:824] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:26.423530 13569 raft_consensus.cc:348] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:26.423650 13569 raft_consensus.cc:374] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:26.423671 13569 raft_consensus.cc:729] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ab8d08f87946c69ceacb3e0097182b, State: Initialized, Role: FOLLOWER
I20260619 16:56:26.423801 13569 consensus_queue.cc:260] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:26.423878 13571 tablet_bootstrap.cc:492] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12: No bootstrap required, opened a new log
I20260619 16:56:26.423944 13571 ts_tablet_manager.cc:1397] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:56:26.424078 13559 heartbeater.cc:503] Master 127.30.148.190:33849 was elected leader, sending a full tablet report...
I20260619 16:56:26.424088 13569 ts_tablet_manager.cc:1428] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 16:56:26.425658 13571 raft_consensus.cc:348] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:26.425768 13571 raft_consensus.cc:374] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:26.425787 13571 raft_consensus.cc:729] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e262d168e0e9460594babcb1b67a2f12, State: Initialized, Role: FOLLOWER
I20260619 16:56:26.425913 13571 consensus_queue.cc:260] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:26.426189 13571 ts_tablet_manager.cc:1428] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 16:56:26.426244 13427 heartbeater.cc:503] Master 127.30.148.190:33849 was elected leader, sending a full tablet report...
W20260619 16:56:26.515456 13428 tablet.cc:2369] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:56:26.609172 13290 tablet.cc:2369] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:56:26.632543 13573 raft_consensus.cc:482] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:56:26.632686 13573 raft_consensus.cc:504] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:26.633095 13573 leader_election.cc:288] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255), 4ed013ce13f643f7a4e667c4e89fb8c2 (127.30.148.129:37441)
I20260619 16:56:26.637073 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.633262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:37441 (local address 127.30.148.131:48907)
0619 16:56:26.633318 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:26.633329 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:26.633398 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:26.633674 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:26.633676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:26.633678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:26.633797 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.633799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.634719 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.634719 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.636054 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.636056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.636818 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.636819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.636886 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.636891 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:56:26.636948 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 16:56:26.637002 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":80}
I20260619 16:56:26.637092 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.633391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:51293 (local address 127.30.148.130:45255)
0619 16:56:26.633451 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:26.633453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:26.633553 (+ 100us) server_negotiation.cc:407] Connection header received
0619 16:56:26.633579 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:26.633580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:26.633589 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:26.633645 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:26.633880 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.634633 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.636142 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.636710 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.636788 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.636797 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:26.636990 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 16:56:26.637033 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:56:26.637089 13575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.633314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:48907 (local address 127.30.148.129:37441)
0619 16:56:26.633432 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:26.633452 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:26.633463 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:56:26.633494 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:26.633499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:26.633510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:26.633612 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:26.633880 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.634627 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.636143 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.636710 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.636789 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.636797 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:26.636963 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 16:56:26.637019 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":66,"threads_started":1}
I20260619 16:56:26.637274 13576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.633272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:45255 (local address 127.30.148.131:51293)
0619 16:56:26.633479 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:26.633496 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:26.633537 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:26.633674 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:26.633676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:26.633678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:26.633797 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.633799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.634719 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.634719 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.636053 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:26.636056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.636810 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.636811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:26.636899 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.636902 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:56:26.637162 (+ 260us) client_negotiation.cc:240] Negotiation successful
0619 16:56:26.637205 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":111,"thread_start_us":107,"threads_started":1}
I20260619 16:56:26.637463 13243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585c5661f24a47d4a638fcc43c3b6f0c" candidate_uuid: "54ab8d08f87946c69ceacb3e0097182b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" is_pre_election: true
I20260619 16:56:26.637624 13243 raft_consensus.cc:2454] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ab8d08f87946c69ceacb3e0097182b in term 0.
I20260619 16:56:26.637722 13381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585c5661f24a47d4a638fcc43c3b6f0c" candidate_uuid: "54ab8d08f87946c69ceacb3e0097182b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e262d168e0e9460594babcb1b67a2f12" is_pre_election: true
I20260619 16:56:26.637830 13446 leader_election.cc:302] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed013ce13f643f7a4e667c4e89fb8c2, 54ab8d08f87946c69ceacb3e0097182b; no voters:
I20260619 16:56:26.637909 13381 raft_consensus.cc:2454] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ab8d08f87946c69ceacb3e0097182b in term 0.
I20260619 16:56:26.637938 13573 raft_consensus.cc:2781] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:56:26.638010 13573 raft_consensus.cc:482] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:56:26.638033 13573 raft_consensus.cc:3037] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:26.638767 13573 raft_consensus.cc:504] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:26.639034 13573 leader_election.cc:288] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [CANDIDATE]: Term 1 election: Requested vote from peers e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255), 4ed013ce13f643f7a4e667c4e89fb8c2 (127.30.148.129:37441)
I20260619 16:56:26.639081 13243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585c5661f24a47d4a638fcc43c3b6f0c" candidate_uuid: "54ab8d08f87946c69ceacb3e0097182b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2"
I20260619 16:56:26.639175 13243 raft_consensus.cc:3037] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:26.640079 13243 raft_consensus.cc:2454] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ab8d08f87946c69ceacb3e0097182b in term 1.
I20260619 16:56:26.640246 13446 leader_election.cc:302] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed013ce13f643f7a4e667c4e89fb8c2, 54ab8d08f87946c69ceacb3e0097182b; no voters:
I20260619 16:56:26.640327 13573 raft_consensus.cc:2781] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:56:26.640480 13573 raft_consensus.cc:686] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 1 LEADER]: Becoming Leader. State: Replica: 54ab8d08f87946c69ceacb3e0097182b, State: Running, Role: LEADER
I20260619 16:56:26.640575 13573 consensus_queue.cc:237] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:26.641460 13381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585c5661f24a47d4a638fcc43c3b6f0c" candidate_uuid: "54ab8d08f87946c69ceacb3e0097182b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e262d168e0e9460594babcb1b67a2f12"
I20260619 16:56:26.641546 13381 raft_consensus.cc:3037] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:26.641427 13100 catalog_manager.cc:5495] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b reported cstate change: term changed from 0 to 1, leader changed from <none> to 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131). New cstate: current_term: 1 leader_uuid: "54ab8d08f87946c69ceacb3e0097182b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } health_report { overall_health: UNKNOWN } } }
I20260619 16:56:26.642489 13381 raft_consensus.cc:2454] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ab8d08f87946c69ceacb3e0097182b in term 1.
W20260619 16:56:26.643908 13560 tablet.cc:2369] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:56:26.697825 13566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:26.695463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40504 (local address 127.30.148.131:39425)
0619 16:56:26.695530 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:26.695532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:26.695541 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:56:26.695669 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:26.695670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:26.695689 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:26.695756 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:56:26.695881 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.696339 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.697056 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:26.697256 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:26.697329 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:26.697511 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:56:26.697604 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:56:26.697688 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:26.697711 (+ 23us) server_negotiation.cc:299] Negotiation successful
0619 16:56:26.697729 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 16:56:26.699499 13243 raft_consensus.cc:1261] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Refusing update from remote peer 54ab8d08f87946c69ceacb3e0097182b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:56:26.699559 13381 raft_consensus.cc:1261] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12 [term 1 FOLLOWER]: Refusing update from remote peer 54ab8d08f87946c69ceacb3e0097182b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:56:26.699862 13573 consensus_queue.cc:1035] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:56:26.699993 13577 consensus_queue.cc:1035] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:56:26.702682 13582 mvcc.cc:205] Tried to move back new op lower bound from 7298614012718739456 to 7298614012480450560. Current Snapshot: MvccSnapshot[applied={T|T < 7298614012718739456}]
I20260619 16:56:26.702930 13580 mvcc.cc:205] Tried to move back new op lower bound from 7298614012718739456 to 7298614012480450560. Current Snapshot: MvccSnapshot[applied={T|T < 7298614012718739456 or (T in {7298614012718739456})}]
I20260619 16:56:26.704418 13101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:56:26.704506 13101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:56:26.705271 13101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Sending DeleteTablet for 3 replicas of tablet 585c5661f24a47d4a638fcc43c3b6f0c
I20260619 16:56:26.705590 13223 tablet_service.cc:1514] Processing DeleteTablet for tablet 585c5661f24a47d4a638fcc43c3b6f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:26 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:36882
I20260619 16:56:26.705580 13493 tablet_service.cc:1514] Processing DeleteTablet for tablet 585c5661f24a47d4a638fcc43c3b6f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:26 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:40496
I20260619 16:56:26.705618 13361 tablet_service.cc:1514] Processing DeleteTablet for tablet 585c5661f24a47d4a638fcc43c3b6f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:26 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:55620
I20260619 16:56:26.705819 13589 tablet_replica.cc:330] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b: stopping tablet replica
I20260619 16:56:26.705914 13589 raft_consensus.cc:2229] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:56:26.705910 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 13139
I20260619 16:56:26.705927 13588 tablet_replica.cc:330] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: stopping tablet replica
I20260619 16:56:26.706025 13589 raft_consensus.cc:2258] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:56:26.706116 13588 raft_consensus.cc:2229] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:56:26.706171 13590 tablet_replica.cc:330] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12: stopping tablet replica
I20260619 16:56:26.706245 13588 raft_consensus.cc:2258] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:56:26.706256 13590 raft_consensus.cc:2229] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:56:26.706350 13590 raft_consensus.cc:2258] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:56:26.706738 13590 ts_tablet_manager.cc:1905] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:56:26.707273 13589 ts_tablet_manager.cc:1905] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:56:26.708420 13590 ts_tablet_manager.cc:1918] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:56:26.708496 13590 log.cc:1196] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/585c5661f24a47d4a638fcc43c3b6f0c
I20260619 16:56:26.708781 13590 ts_tablet_manager.cc:1939] T 585c5661f24a47d4a638fcc43c3b6f0c P e262d168e0e9460594babcb1b67a2f12: Deleting consensus metadata
I20260619 16:56:26.709189 13589 ts_tablet_manager.cc:1918] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:56:26.709177 13088 catalog_manager.cc:4841] TS e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255): tablet 585c5661f24a47d4a638fcc43c3b6f0c (table test-table [id=510ad7d48ff44ccd8aad0cc72ab07344]) successfully deleted
I20260619 16:56:26.709326 13589 log.cc:1196] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/585c5661f24a47d4a638fcc43c3b6f0c
I20260619 16:56:26.709664 13589 ts_tablet_manager.cc:1939] T 585c5661f24a47d4a638fcc43c3b6f0c P 54ab8d08f87946c69ceacb3e0097182b: Deleting consensus metadata
I20260619 16:56:26.710026 13088 catalog_manager.cc:4841] TS 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425): tablet 585c5661f24a47d4a638fcc43c3b6f0c (table test-table [id=510ad7d48ff44ccd8aad0cc72ab07344]) successfully deleted
W20260619 16:56:26.713411 13085 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:37441 (error 108)
W20260619 16:56:26.713889 13565 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:56:26.713759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:37441 (local address 127.0.0.1:36890)
0619 16:56:26.713816 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:26.713850 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:37441: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 16:56:26.714123 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 13294
W20260619 16:56:26.714105 13085 catalog_manager.cc:4584] TS 4ed013ce13f643f7a4e667c4e89fb8c2 (127.30.148.129:37441): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 585c5661f24a47d4a638fcc43c3b6f0c: Network error: Client connection negotiation failed: client connection to 127.30.148.129:37441: connect: Connection reset by peer (error 104)
I20260619 16:56:26.719084 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 13430
I20260619 16:56:26.723201 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 13068
I20260619 16:56:26.727492 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33849
--webserver_interface=127.30.148.190
--webserver_port=35757
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:33849
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:26.804747 13592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:26.804958 13592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:26.804992 13592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:26.806084 13592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:56:26.806149 13592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:26.806173 13592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:26.806195 13592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:56:26.806221 13592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:56:26.807631 13592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:33849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=35757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:26.807922 13592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:26.808140 13592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:26.812695 13600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:26.812726 13597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:26.812744 13592 server_base.cc:1034] running on GCE node
W20260619 16:56:26.812716 13598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:26.813050 13592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:26.813319 13592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:26.814491 13592 hybrid_clock.cc:648] HybridClock initialized: now 1781888186814479 us; error 31 us; skew 500 ppm
Jun 19 16:56:26 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888186, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:26.816735 13592 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:56:26.817178 13592 webserver.cc:460] Webserver started at http://127.30.148.190:35757/ using document root <none> and password file <none>
I20260619 16:56:26.817445 13592 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:26.817512 13592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:26.818732 13592 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:26.819463 13607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:26.819711 13592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 16:56:26.819828 13592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "c9c97365988242ebb7523edbabf45a54"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "e0a1620d343539ae93d4f69b790e950f"
server_key_iv: "e9eb3430fb5fa703c1b550383d893d8c"
server_key_version: "encryptionkey@0"
I20260619 16:56:26.820094 13592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:26.833982 13592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:26.835568 13592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:26.835664 13592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:26.839486 13592 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:33849
I20260619 16:56:26.839550 13659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:33849 every 8 connection(s)
I20260619 16:56:26.839864 13592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:56:26.841229 13660 sys_catalog.cc:263] Verifying existing consensus state
I20260619 16:56:26.841746 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 13592
I20260619 16:56:26.841897 13660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Bootstrap starting.
I20260619 16:56:26.842043 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:37441
--local_ip_for_outbound_sockets=127.30.148.129
--tserver_master_addrs=127.30.148.190:33849
--webserver_port=45733
--webserver_interface=127.30.148.129
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:56:26.850203 13660 log.cc:824] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:26.852231 13660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:56:26.852505 13660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54: Bootstrap complete.
I20260619 16:56:26.854643 13660 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } }
I20260619 16:56:26.854869 13660 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9c97365988242ebb7523edbabf45a54, State: Initialized, Role: FOLLOWER
I20260619 16:56:26.855060 13660 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } }
I20260619 16:56:26.855152 13660 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:56:26.855204 13660 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:56:26.855249 13660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 2 FOLLOWER]: Advancing to term 3
I20260619 16:56:26.856149 13660 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } }
I20260619 16:56:26.856258 13660 leader_election.cc:302] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9c97365988242ebb7523edbabf45a54; no voters:
I20260619 16:56:26.856413 13660 leader_election.cc:288] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 16:56:26.856460 13664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 16:56:26.856578 13664 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [term 3 LEADER]: Becoming Leader. State: Replica: c9c97365988242ebb7523edbabf45a54, State: Running, Role: LEADER
I20260619 16:56:26.856652 13660 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:56:26.856684 13664 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } }
I20260619 16:56:26.856961 13664 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9c97365988242ebb7523edbabf45a54. Latest consensus state: current_term: 3 leader_uuid: "c9c97365988242ebb7523edbabf45a54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } } }
I20260619 16:56:26.857133 13664 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:26.856962 13665 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c9c97365988242ebb7523edbabf45a54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c97365988242ebb7523edbabf45a54" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33849 } } }
I20260619 16:56:26.857432 13673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:56:26.857450 13665 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9c97365988242ebb7523edbabf45a54 [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:26.858155 13673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:56:26.858253 13673 catalog_manager.cc:1259] Loaded cluster ID: 8243259d3b32474fad31d258453face3
I20260619 16:56:26.858296 13673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 16:56:26.858518 13673 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2dd65f69ad google::LogMessage::Fail() at ??:0
@ 0x7f2dd65f988a google::LogMessage::SendToLog() at ??:0
@ 0x7f2dd65f64de google::LogMessage::Flush() at ??:0
@ 0x7f2dd65f7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2dde51b0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f2dde5248b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2dd84c40d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2dd84b9954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2dd931d6db start_thread at ??:0
@ 0x7f2dd50f461f clone at ??:0
W20260619 16:56:26.932502 13662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:26.932672 13662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:26.932693 13662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:26.933871 13662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:26.933910 13662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:26.933936 13662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:56:26.935338 13662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:37441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=45733
--tserver_master_addrs=127.30.148.190:33849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:26.935544 13662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:26.935802 13662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:26.941684 13697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:26.941671 13699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:26.941704 13696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:26.941910 13662 server_base.cc:1034] running on GCE node
I20260619 16:56:26.942194 13662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:26.942456 13662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:26.943606 13662 hybrid_clock.cc:648] HybridClock initialized: now 1781888186943598 us; error 26 us; skew 500 ppm
Jun 19 16:56:26 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888186, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:26.946205 13662 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:56:26.946604 13662 webserver.cc:460] Webserver started at http://127.30.148.129:45733/ using document root <none> and password file <none>
I20260619 16:56:26.946834 13662 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:26.946903 13662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:26.948232 13662 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:56:26.948944 13706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:26.949119 13662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:56:26.949237 13662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "4ed013ce13f643f7a4e667c4e89fb8c2"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "7532aeb2205bb4694128b46e0e0904f2"
server_key_iv: "8875a8894a4ce085f9789336c63a1fc5"
server_key_version: "encryptionkey@0"
I20260619 16:56:26.949716 13662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:26.966427 13662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:26.968576 13662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:26.968732 13662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:26.968940 13662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:26.969426 13713 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 16:56:26.971259 13662 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 16:56:26.971318 13662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 16:56:26.971381 13662 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 16:56:26.971966 13662 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 16:56:26.972021 13662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 16:56:26.972082 13713 tablet_bootstrap.cc:492] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: Bootstrap starting.
I20260619 16:56:26.979300 13662 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:37441
I20260619 16:56:26.979434 13820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:37441 every 8 connection(s)
I20260619 16:56:26.979703 13662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:56:26.982905 13713 log.cc:824] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:26.985442 13713 tablet_bootstrap.cc:492] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:56:26.985741 13713 tablet_bootstrap.cc:492] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: Bootstrap complete.
I20260619 16:56:26.986953 13713 ts_tablet_manager.cc:1397] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.006s
I20260619 16:56:26.987622 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 13662
I20260619 16:56:26.987875 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:45255
--local_ip_for_outbound_sockets=127.30.148.130
--tserver_master_addrs=127.30.148.190:33849
--webserver_port=41331
--webserver_interface=127.30.148.130
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:56:26.988354 13713 raft_consensus.cc:348] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:26.988679 13713 raft_consensus.cc:729] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed013ce13f643f7a4e667c4e89fb8c2, State: Initialized, Role: FOLLOWER
I20260619 16:56:26.988838 13713 consensus_queue.cc:260] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:26.989118 13713 ts_tablet_manager.cc:1428] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260619 16:56:27.035351 13823 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:56:26.980355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:39469)
0619 16:56:26.980518 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:26.980559 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:26.981007 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:27.035168 (+ 54161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
W20260619 16:56:27.035593 13821 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:33849 (0 consecutive failures): Network error: Failed to ping master at 127.30.148.190:33849: Client connection negotiation failed: client connection to 127.30.148.190:33849: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 16:56:27.035923 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:27.035803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:55177)
0619 16:56:27.035863 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:27.035902 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 16:56:27.036372 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:27.036292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:39917)
0619 16:56:27.036331 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:27.036355 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 16:56:27.036490 13821 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 16:56:27.067983 13825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:27.068159 13825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:27.068181 13825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:27.069227 13825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:27.069263 13825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:27.069288 13825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:56:27.070834 13825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:45255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=41331
--tserver_master_addrs=127.30.148.190:33849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:27.071077 13825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:27.071266 13825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:56:27.076103 13825 server_base.cc:1034] running on GCE node
W20260619 16:56:27.076110 13832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:27.076081 13831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:27.076087 13834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:27.076530 13825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:27.076789 13825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:27.077934 13825 hybrid_clock.cc:648] HybridClock initialized: now 1781888187077912 us; error 38 us; skew 500 ppm
Jun 19 16:56:27 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888187, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:27.080220 13825 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:56:27.080665 13825 webserver.cc:460] Webserver started at http://127.30.148.130:41331/ using document root <none> and password file <none>
I20260619 16:56:27.080874 13825 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:27.080948 13825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:27.082229 13825 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:27.082791 13841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:27.082950 13825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:56:27.083045 13825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "e262d168e0e9460594babcb1b67a2f12"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "52c14837f25bf91aa62238dfd08708fc"
server_key_iv: "6d56b9051313adbf7e852bb42f213a5a"
server_key_version: "encryptionkey@0"
I20260619 16:56:27.083344 13825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:27.114250 13825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:27.115927 13825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:27.116151 13825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:27.116381 13825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:27.116677 13825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:27.116730 13825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:27.116770 13825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:27.116797 13825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:27.122479 13825 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:45255
I20260619 16:56:27.122526 13954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:45255 every 8 connection(s)
I20260619 16:56:27.122877 13825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:56:27.123890 13957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:27.123539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:50599)
0619 16:56:27.123692 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:27.123761 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
W20260619 16:56:27.124280 13955 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:33849 (0 consecutive failures): Network error: Failed to ping master at 127.30.148.190:33849: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
I20260619 16:56:27.124529 13957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:27.124448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:51451)
0619 16:56:27.124487 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:27.124513 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 16:56:27.124773 13957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:27.124705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:51527)
0619 16:56:27.124740 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:27.124760 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260619 16:56:27.124882 13955 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 16:56:27.133061 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 13825
I20260619 16:56:27.133332 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:39425
--local_ip_for_outbound_sockets=127.30.148.131
--tserver_master_addrs=127.30.148.190:33849
--webserver_port=40259
--webserver_interface=127.30.148.131
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:44867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:27.214097 13958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:27.214342 13958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:27.214378 13958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:27.216001 13958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:27.216045 13958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:27.216092 13958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:56:27.217846 13958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:44867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:39425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=40259
--tserver_master_addrs=127.30.148.190:33849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:27.218060 13958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:27.218287 13958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:27.222937 13963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:27.222930 13966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:27.222982 13964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:27.223273 13958 server_base.cc:1034] running on GCE node
I20260619 16:56:27.223450 13958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:27.223668 13958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:27.224805 13958 hybrid_clock.cc:648] HybridClock initialized: now 1781888187224794 us; error 27 us; skew 500 ppm
Jun 19 16:56:27 dist-test-slave-xb9n krb5kdc[12524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888187, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:27.232103 13958 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:56:27.232559 13958 webserver.cc:460] Webserver started at http://127.30.148.131:40259/ using document root <none> and password file <none>
I20260619 16:56:27.232769 13958 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:27.232811 13958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:27.234167 13958 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260619 16:56:27.234746 13973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:27.234922 13958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:56:27.235008 13958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "54ab8d08f87946c69ceacb3e0097182b"
format_stamp: "Formatted at 2026-06-19 16:56:25 on dist-test-slave-xb9n"
server_key: "a1af613bebcb59c6a11ad122450afe8e"
server_key_iv: "05ca71285e516d47d2a826f40642107b"
server_key_version: "encryptionkey@0"
I20260619 16:56:27.235316 13958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:27.245699 13958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:27.247426 13958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:27.247570 13958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:27.247804 13958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:27.248102 13958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:27.248135 13958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:27.248174 13958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:27.248201 13958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:27.254073 13958 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:39425
I20260619 16:56:27.254122 14086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:39425 every 8 connection(s)
I20260619 16:56:27.254427 13958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:56:27.255355 14089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:27.255030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:51973)
0619 16:56:27.255159 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:27.255231 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
W20260619 16:56:27.255716 14087 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:33849 (0 consecutive failures): Network error: Failed to ping master at 127.30.148.190:33849: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
I20260619 16:56:27.255993 14089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:27.255901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:52815)
0619 16:56:27.255950 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:27.255976 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 16:56:27.256285 14089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:27.256210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:40915)
0619 16:56:27.256251 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:27.256272 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 16:56:27.256398 14087 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 16:56:27.257767 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 13958
I20260619 16:56:28.037173 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:28.036932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:49105)
0619 16:56:28.037064 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:28.037102 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":59,"threads_started":1}
I20260619 16:56:28.125455 14092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:28.125203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:49537)
0619 16:56:28.125321 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:28.125397 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260619 16:56:28.220932 14093 raft_consensus.cc:482] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:56:28.221086 14093 raft_consensus.cc:504] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:28.221530 14093 leader_election.cc:288] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425), e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255)
Jun 19 16:56:28 dist-test-slave-xb9n krb5kdc[12524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888186, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.130@KRBTEST.COM
Jun 19 16:56:28 dist-test-slave-xb9n krb5kdc[12524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888186, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.131@KRBTEST.COM
I20260619 16:56:28.233888 14096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:28.221910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:55781 (local address 127.30.148.131:39425)
0619 16:56:28.222088 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:28.222100 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:28.222123 (+ 23us) server_negotiation.cc:407] Connection header received
0619 16:56:28.222661 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:28.222664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:28.222732 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:28.222903 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:28.224179 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:28.225487 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:28.226850 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:28.227404 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:28.227526 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:28.230399 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:28.230403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:28.230403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:28.230415 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:28.232266 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:28.232666 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:28.232668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:28.232669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:28.232755 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:28.232996 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:28.232997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:28.232997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:28.233225 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:28.233325 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:28.233657 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 16:56:28.233778 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260619 16:56:28.233888 14094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:28.221663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:39425 (local address 127.30.148.129:55781)
0619 16:56:28.221904 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:28.221916 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:28.221985 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:28.222949 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:28.222953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:28.224000 (+ 1047us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:28.224064 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:28.224066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:28.225585 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:28.225586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:28.226754 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:28.226756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:28.227523 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:28.227525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:28.227642 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:28.228254 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:28.228261 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:28.230289 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:28.232372 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:28.232373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:28.232373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:28.232569 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:28.232861 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:28.232862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:28.232862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:28.232926 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:28.233308 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:28.233314 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:28.233464 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 16:56:28.233588 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 16:56:28.233790 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":13}
I20260619 16:56:28.233992 14095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:28.221860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:35033 (local address 127.30.148.130:45255)
0619 16:56:28.222156 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:28.222163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:28.222189 (+ 26us) server_negotiation.cc:407] Connection header received
0619 16:56:28.222679 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:28.222682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:28.222731 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:28.223042 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:28.223882 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:28.225305 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:28.226851 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:28.227526 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:28.227678 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:28.230165 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:28.230168 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:28.230169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:28.230179 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:28.232129 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:28.232611 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:28.232614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:28.232614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:28.232710 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:28.232976 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:28.232977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:28.232978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:28.233225 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:28.233319 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:28.233676 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 16:56:28.233824 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":72,"threads_started":1}
I20260619 16:56:28.234007 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:28.221781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:45255 (local address 127.30.148.129:35033)
0619 16:56:28.221910 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:28.221922 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:28.221984 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:28.222949 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:28.222952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:28.223436 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:28.223757 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:28.223766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:28.225437 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:28.225438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:28.226753 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:28.226755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:28.227824 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:28.227825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:28.227885 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:28.228421 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:28.228426 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:28.230023 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:28.232242 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:28.232249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:28.232251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:28.232503 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:28.232789 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:28.232789 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:28.232790 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:28.232902 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:28.233314 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:28.233315 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:28.233466 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 16:56:28.233588 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 16:56:28.233874 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":175,"thread_start_us":55,"threads_started":1}
W20260619 16:56:28.235378 13710 leader_election.cc:341] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255): Not found: Tablet not found: 585c5661f24a47d4a638fcc43c3b6f0c
W20260619 16:56:28.235462 13710 leader_election.cc:341] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425): Not found: Tablet not found: 585c5661f24a47d4a638fcc43c3b6f0c
I20260619 16:56:28.235487 13710 leader_election.cc:302] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed013ce13f643f7a4e667c4e89fb8c2; no voters: 54ab8d08f87946c69ceacb3e0097182b, e262d168e0e9460594babcb1b67a2f12
I20260619 16:56:28.235589 14093 raft_consensus.cc:2726] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 16:56:28.256852 14097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:28.256685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:44599)
0619 16:56:28.256784 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:28.256821 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":49,"threads_started":1}
I20260619 16:56:29.037948 14098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:29.037716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:39215)
0619 16:56:29.037860 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:29.037894 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
I20260619 16:56:29.126040 14099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:29.125845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:48399)
0619 16:56:29.125967 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:29.125999 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260619 16:56:29.257495 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:29.257266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:55965)
0619 16:56:29.257428 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:29.257461 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":52,"threads_started":1}
I20260619 16:56:30.038720 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:30.038479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:60069)
0619 16:56:30.038632 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:30.038672 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260619 16:56:30.126605 14102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:30.126421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:33325)
0619 16:56:30.126539 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:30.126570 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260619 16:56:30.177070 14103 raft_consensus.cc:482] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:56:30.177158 14103 raft_consensus.cc:504] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:30.177392 14103 leader_election.cc:288] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425), e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255)
W20260619 16:56:30.177793 13710 leader_election.cc:341] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425): Not found: Tablet not found: 585c5661f24a47d4a638fcc43c3b6f0c
W20260619 16:56:30.177852 13710 leader_election.cc:341] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255): Not found: Tablet not found: 585c5661f24a47d4a638fcc43c3b6f0c
I20260619 16:56:30.177868 13710 leader_election.cc:302] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed013ce13f643f7a4e667c4e89fb8c2; no voters: 54ab8d08f87946c69ceacb3e0097182b, e262d168e0e9460594babcb1b67a2f12
I20260619 16:56:30.177960 14103 raft_consensus.cc:2726] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 16:56:30.258106 14104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:30.257924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:53807)
0619 16:56:30.258043 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:30.258074 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260619 16:56:31.039393 14105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:31.039193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:38563)
0619 16:56:31.039313 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:31.039346 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":56,"threads_started":1}
I20260619 16:56:31.127249 14106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:31.127050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:44081)
0619 16:56:31.127176 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:31.127212 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":55,"threads_started":1}
I20260619 16:56:31.258699 14107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:31.258518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:52767)
0619 16:56:31.258633 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:31.258666 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":55,"threads_started":1}
I20260619 16:56:32.040117 14108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:32.039895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:47855)
0619 16:56:32.040034 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:32.040069 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":56,"threads_started":1}
I20260619 16:56:32.128010 14109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:32.127793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:41317)
0619 16:56:32.127942 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:32.127976 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260619 16:56:32.259358 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:32.259155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:41561)
0619 16:56:32.259275 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:32.259320 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260619 16:56:32.502202 14111 raft_consensus.cc:482] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:56:32.502311 14111 raft_consensus.cc:504] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:32.502471 14111 leader_election.cc:288] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425), e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255)
W20260619 16:56:32.502873 13710 leader_election.cc:341] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425): Not found: Tablet not found: 585c5661f24a47d4a638fcc43c3b6f0c
W20260619 16:56:32.502945 13710 leader_election.cc:341] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255): Not found: Tablet not found: 585c5661f24a47d4a638fcc43c3b6f0c
I20260619 16:56:32.502960 13710 leader_election.cc:302] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed013ce13f643f7a4e667c4e89fb8c2; no voters: 54ab8d08f87946c69ceacb3e0097182b, e262d168e0e9460594babcb1b67a2f12
I20260619 16:56:32.503077 14111 raft_consensus.cc:2726] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 16:56:33.040895 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:33.040661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:35639)
0619 16:56:33.040807 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:33.040844 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260619 16:56:33.128613 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:33.128430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:42169)
0619 16:56:33.128545 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:33.128574 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260619 16:56:33.259959 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:33.259787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:54427)
0619 16:56:33.259890 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:33.259923 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
I20260619 16:56:34.041640 14115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:34.041343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:45553)
0619 16:56:34.041558 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:34.041594 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":122,"threads_started":1}
I20260619 16:56:34.129228 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:34.129045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:58437)
0619 16:56:34.129158 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:34.129192 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260619 16:56:34.260666 14117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:34.260453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:48641)
0619 16:56:34.260598 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:34.260632 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260619 16:56:35.042277 14118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:35.042072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:43537)
0619 16:56:35.042202 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:35.042232 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260619 16:56:35.129876 14119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:35.129683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:50737)
0619 16:56:35.129800 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:35.129835 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260619 16:56:35.261469 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:35.261236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:36433)
0619 16:56:35.261350 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:35.261432 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260619 16:56:36.042923 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:36.042692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:59283)
0619 16:56:36.042843 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:36.042879 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
I20260619 16:56:36.130436 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:36.130267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:44793)
0619 16:56:36.130371 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:36.130402 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":53,"threads_started":1}
I20260619 16:56:36.262100 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:36.261887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:45953)
0619 16:56:36.262029 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:36.262065 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260619 16:56:36.471619 14124 raft_consensus.cc:482] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:56:36.471710 14124 raft_consensus.cc:504] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:36.471895 14124 leader_election.cc:288] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425), e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255)
W20260619 16:56:36.472251 13710 leader_election.cc:341] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425): Not found: Tablet not found: 585c5661f24a47d4a638fcc43c3b6f0c
W20260619 16:56:36.472333 13710 leader_election.cc:341] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255): Not found: Tablet not found: 585c5661f24a47d4a638fcc43c3b6f0c
I20260619 16:56:36.472371 13710 leader_election.cc:302] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed013ce13f643f7a4e667c4e89fb8c2; no voters: 54ab8d08f87946c69ceacb3e0097182b, e262d168e0e9460594babcb1b67a2f12
I20260619 16:56:36.472467 14124 raft_consensus.cc:2726] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 16:56:37.043533 14125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:37.043354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:48235)
0619 16:56:37.043463 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:37.043494 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":53,"threads_started":1}
I20260619 16:56:37.130949 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:37.130772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:49809)
0619 16:56:37.130887 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:37.130916 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":55,"threads_started":1}
I20260619 16:56:37.262764 14127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:37.262535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:35911)
0619 16:56:37.262688 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:37.262725 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":71,"threads_started":1}
I20260619 16:56:38.044170 14128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:38.043891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:47951)
0619 16:56:38.044093 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:38.044127 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":56,"threads_started":1}
I20260619 16:56:38.131696 14129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:38.131482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:54569)
0619 16:56:38.131624 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:38.131659 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":55,"threads_started":1}
I20260619 16:56:38.263455 14130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:38.263275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:57741)
0619 16:56:38.263387 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:38.263420 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
I20260619 16:56:39.044919 14131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:39.044686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:51029)
0619 16:56:39.044837 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:39.044873 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":57,"threads_started":1}
I20260619 16:56:39.132321 14132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:39.132136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:43251)
0619 16:56:39.132248 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:39.132282 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
I20260619 16:56:39.264145 14133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:39.263956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:58797)
0619 16:56:39.264079 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:39.264109 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260619 16:56:40.045619 14134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:40.045332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:41959)
0619 16:56:40.045539 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:40.045571 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":149,"threads_started":1}
I20260619 16:56:40.132952 14135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:40.132750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:46347)
0619 16:56:40.132882 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:40.132916 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260619 16:56:40.264768 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:40.264582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:48549)
0619 16:56:40.264689 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:40.264718 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":52,"threads_started":1}
I20260619 16:56:41.033591 14137 raft_consensus.cc:482] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:56:41.033707 14137 raft_consensus.cc:504] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ab8d08f87946c69ceacb3e0097182b" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39425 } } peers { permanent_uuid: "e262d168e0e9460594babcb1b67a2f12" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45255 } } peers { permanent_uuid: "4ed013ce13f643f7a4e667c4e89fb8c2" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 37441 } }
I20260619 16:56:41.033910 14137 leader_election.cc:288] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425), e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255)
W20260619 16:56:41.034262 13710 leader_election.cc:341] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54ab8d08f87946c69ceacb3e0097182b (127.30.148.131:39425): Not found: Tablet not found: 585c5661f24a47d4a638fcc43c3b6f0c
W20260619 16:56:41.034345 13710 leader_election.cc:341] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e262d168e0e9460594babcb1b67a2f12 (127.30.148.130:45255): Not found: Tablet not found: 585c5661f24a47d4a638fcc43c3b6f0c
I20260619 16:56:41.034363 13710 leader_election.cc:302] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ed013ce13f643f7a4e667c4e89fb8c2; no voters: 54ab8d08f87946c69ceacb3e0097182b, e262d168e0e9460594babcb1b67a2f12
I20260619 16:56:41.034475 14137 raft_consensus.cc:2726] T 585c5661f24a47d4a638fcc43c3b6f0c P 4ed013ce13f643f7a4e667c4e89fb8c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 16:56:41.046386 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:41.046152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:54955)
0619 16:56:41.046314 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:41.046345 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260619 16:56:41.133603 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:41.133350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:45551)
0619 16:56:41.133529 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:41.133564 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":77,"threads_started":1}
I20260619 16:56:41.265434 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:41.265175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:38489)
0619 16:56:41.265319 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:41.265382 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260619 16:56:42.047251 14141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:42.046986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.129:44083)
0619 16:56:42.047138 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:42.047195 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":71,"threads_started":1}
I20260619 16:56:42.134240 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:42.134053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.130:48513)
0619 16:56:42.134174 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:42.134204 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":54,"threads_started":1}
I20260619 16:56:42.266342 14143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:42.265926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33849 (local address 127.30.148.131:60253)
0619 16:56:42.266229 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:42.266273 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:33849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":67,"threads_started":1}
I20260619 16:56:42.267517 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 13662
I20260619 16:56:42.272596 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 13825
I20260619 16:56:42.276814 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 13958
2026-06-19T16:56:42Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18373 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:56:42 dist-test-slave-xb9n krb5kdc[14146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:56:42 dist-test-slave-xb9n krb5kdc[14146](info): set up 2 sockets
Jun 19 16:56:42 dist-test-slave-xb9n krb5kdc[14146](info): commencing operation
krb5kdc: starting...
W20260619 16:56:43.321708 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:56:43 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:56:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:56:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:56:43.431793 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36793
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:36793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:43.508862 14159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:43.509094 14159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:43.509132 14159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:43.510262 14159 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 16:56:43.510327 14159 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 16:56:43.510349 14159 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 16:56:43.510365 14159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:56:43.510399 14159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:43.510427 14159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:43.510450 14159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:56:43.510481 14159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:56:43.511911 14159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:36793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:43.512179 14159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:43.512388 14159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:43.517319 14164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:43.517386 14167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:43.517407 14165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:43.517331 14159 server_base.cc:1034] running on GCE node
I20260619 16:56:43.517735 14159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:43.517951 14159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:43.519090 14159 hybrid_clock.cc:648] HybridClock initialized: now 1781888203519078 us; error 28 us; skew 500 ppm
Jun 19 16:56:43 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888203, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:43.521355 14159 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:56:43.521884 14159 webserver.cc:460] Webserver started at http://127.30.148.190:42921/ using document root <none> and password file <none>
I20260619 16:56:43.522101 14159 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:43.522164 14159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:43.522310 14159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:56:43.523283 14159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "d97b84b43b26421eb17dc43b5df1ecbc"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "a2f1ea3f9319263b6e6152d3e2ca4094"
server_key_iv: "fb009932bd70e217cc9e636919a3558b"
server_key_version: "encryptionkey@0"
I20260619 16:56:43.523670 14159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "d97b84b43b26421eb17dc43b5df1ecbc"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "a2f1ea3f9319263b6e6152d3e2ca4094"
server_key_iv: "fb009932bd70e217cc9e636919a3558b"
server_key_version: "encryptionkey@0"
I20260619 16:56:43.524935 14159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 16:56:43.525743 14174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:43.525929 14159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:43.526023 14159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "d97b84b43b26421eb17dc43b5df1ecbc"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "a2f1ea3f9319263b6e6152d3e2ca4094"
server_key_iv: "fb009932bd70e217cc9e636919a3558b"
server_key_version: "encryptionkey@0"
I20260619 16:56:43.526126 14159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:43.538719 14159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:43.540412 14159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:43.540565 14159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:43.544417 14159 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:36793
I20260619 16:56:43.544476 14226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:36793 every 8 connection(s)
I20260619 16:56:43.544809 14159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:56:43.545681 14227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:43.545743 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 14159
I20260619 16:56:43.545868 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:56:43.546077 31314 external_mini_cluster.cc:1428] Setting key 88dbc015b9330c11444b78f9c8e06abe
I20260619 16:56:43.548851 14227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Bootstrap starting.
I20260619 16:56:43.549610 14227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:43.549911 14227 log.cc:824] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:43.550552 14227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: No bootstrap required, opened a new log
I20260619 16:56:43.552289 14227 raft_consensus.cc:348] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } }
I20260619 16:56:43.552414 14227 raft_consensus.cc:374] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:43.552436 14227 raft_consensus.cc:729] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97b84b43b26421eb17dc43b5df1ecbc, State: Initialized, Role: FOLLOWER
Jun 19 16:56:43 dist-test-slave-xb9n krb5kdc[14146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:43.552587 14227 consensus_queue.cc:260] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } }
I20260619 16:56:43.552680 14227 raft_consensus.cc:388] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:56:43.552711 14227 raft_consensus.cc:482] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:56:43.552762 14227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:43.553424 14227 raft_consensus.cc:504] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } }
I20260619 16:56:43.553555 14227 leader_election.cc:302] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d97b84b43b26421eb17dc43b5df1ecbc; no voters:
I20260619 16:56:43.553732 14227 leader_election.cc:288] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:56:43.553797 14232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:56:43.553932 14232 raft_consensus.cc:686] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 1 LEADER]: Becoming Leader. State: Replica: d97b84b43b26421eb17dc43b5df1ecbc, State: Running, Role: LEADER
I20260619 16:56:43.553984 14227 sys_catalog.cc:564] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:56:43.554280 14232 consensus_queue.cc:237] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } }
I20260619 16:56:43.555819 14234 sys_catalog.cc:455] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d97b84b43b26421eb17dc43b5df1ecbc. Latest consensus state: current_term: 1 leader_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } } }
I20260619 16:56:43.555992 14234 sys_catalog.cc:458] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: This master's current role is: LEADER
I20260619 16:56:43.555953 14233 sys_catalog.cc:455] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } } }
I20260619 16:56:43.556255 14233 sys_catalog.cc:458] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: This master's current role is: LEADER
W20260619 16:56:43.556327 14247 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:56:43.556381 14247 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:56:43.556433 14248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:56:43.556581 14248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:56:43.557096 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:43.546578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36962 (local address 127.30.148.190:36793)
0619 16:56:43.546746 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:43.546753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:43.546776 (+ 23us) server_negotiation.cc:407] Connection header received
0619 16:56:43.547208 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:43.547214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:43.547256 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:43.547359 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:43.548166 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:43.549039 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:43.550202 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:43.550626 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:43.550739 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:43.553084 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:43.553087 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:43.553088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:43.553097 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:43.555168 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:43.555522 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:43.555525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:43.555526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:43.555647 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:43.555836 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:43.555836 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:43.555837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:43.556152 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:43.556331 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:43.556771 (+ 440us) server_negotiation.cc:299] Negotiation successful
0619 16:56:43.556903 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260619 16:56:43.558125 14248 catalog_manager.cc:1347] Generated new cluster ID: c9895b0ec9e94ddb8b834f839110174b
I20260619 16:56:43.558175 14248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:56:43.573520 14248 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:56:43.574023 14248 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:56:43.577513 14248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Generated new TSK 0
I20260619 16:56:43.577677 14248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:56:43.607772 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36793
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:43.686754 14253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:43.686995 14253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:43.687032 14253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:43.688177 14253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:43.688246 14253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:43.688290 14253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:56:43.689726 14253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:43.689996 14253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:43.690213 14253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:43.694988 14261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:43.695027 14259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:43.695155 14253 server_base.cc:1034] running on GCE node
W20260619 16:56:43.694988 14258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:43.695545 14253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:43.695778 14253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:43.696910 14253 hybrid_clock.cc:648] HybridClock initialized: now 1781888203696901 us; error 26 us; skew 500 ppm
Jun 19 16:56:43 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888203, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:43.699370 14253 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:56:43.699810 14253 webserver.cc:460] Webserver started at http://127.30.148.129:45717/ using document root <none> and password file <none>
I20260619 16:56:43.700045 14253 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:43.700114 14253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:43.700263 14253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:56:43.701231 14253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "18aa84a3feda4c939c4b5282191eeef5"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "50bc5059e85298507082ba4714333fd1"
server_key_iv: "1278d3db870df7718b0e9665921080f5"
server_key_version: "encryptionkey@0"
I20260619 16:56:43.701609 14253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "18aa84a3feda4c939c4b5282191eeef5"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "50bc5059e85298507082ba4714333fd1"
server_key_iv: "1278d3db870df7718b0e9665921080f5"
server_key_version: "encryptionkey@0"
I20260619 16:56:43.702912 14253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:43.703625 14268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:43.703795 14253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:43.703912 14253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "18aa84a3feda4c939c4b5282191eeef5"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "50bc5059e85298507082ba4714333fd1"
server_key_iv: "1278d3db870df7718b0e9665921080f5"
server_key_version: "encryptionkey@0"
I20260619 16:56:43.703985 14253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:43.737728 14253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:43.740012 14253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:43.740173 14253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:43.740409 14253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:43.740713 14253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:43.740768 14253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:43.740808 14253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:43.740835 14253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:43.746582 14253 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:36209
I20260619 16:56:43.746622 14381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:36209 every 8 connection(s)
I20260619 16:56:43.746968 14253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
Jun 19 16:56:43 dist-test-slave-xb9n krb5kdc[14146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888203, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:43.753010 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 14253
I20260619 16:56:43.753104 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:56:43.753312 31314 external_mini_cluster.cc:1428] Setting key 7a967a73c278b27a5aa8906d3e1915fb
I20260619 16:56:43.756330 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:43.747670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:54133 (local address 127.30.148.190:36793)
0619 16:56:43.747727 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:43.747729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:43.747943 (+ 214us) server_negotiation.cc:407] Connection header received
0619 16:56:43.748467 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:43.748468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:43.748481 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:43.748527 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:43.749476 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:43.749988 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:43.750972 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:43.751177 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:43.751265 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:43.753399 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:43.753400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:43.753400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:43.753408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:43.754862 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:43.755303 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:43.755304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:43.755305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:43.755368 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:43.755625 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:43.755626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:43.755626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:43.755761 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:43.755824 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:43.756156 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 16:56:43.756240 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 16:56:43.756531 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:43.747659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:54133)
0619 16:56:43.747793 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:43.747835 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:43.748354 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:43.748601 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:43.748604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:43.749142 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:43.749322 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:43.749327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:43.750087 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:43.750088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:43.750868 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:43.750870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:43.751273 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:43.751274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:43.751401 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:43.751829 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:43.751839 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:43.753267 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:43.754951 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:43.754958 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:43.754960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:43.755226 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:43.755438 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:43.755439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:43.755439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:43.755555 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:43.755858 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:43.755863 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:43.756027 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 16:56:43.756134 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 16:56:43.756283 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
I20260619 16:56:43.756992 14382 heartbeater.cc:347] Connected to a master server at 127.30.148.190:36793
I20260619 16:56:43.757079 14382 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:43.757273 14382 heartbeater.cc:511] Master 127.30.148.190:36793 requested a full tablet report, sending...
I20260619 16:56:43.757851 14191 ts_manager.cc:194] Registered new tserver with Master: 18aa84a3feda4c939c4b5282191eeef5 (127.30.148.129:36209)
I20260619 16:56:43.758713 14191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:54133
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:56:43.777005 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36793
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:43.855579 14387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:43.855758 14387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:43.855780 14387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:43.856889 14387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:43.856930 14387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:43.856956 14387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:56:43.858299 14387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:43.858460 14387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:43.858613 14387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:43.863481 14395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:43.863474 14392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:43.863500 14387 server_base.cc:1034] running on GCE node
W20260619 16:56:43.863474 14393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:43.863806 14387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:43.864035 14387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:43.865163 14387 hybrid_clock.cc:648] HybridClock initialized: now 1781888203865144 us; error 37 us; skew 500 ppm
Jun 19 16:56:43 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888203, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:43.867600 14387 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:56:43.868027 14387 webserver.cc:460] Webserver started at http://127.30.148.130:35793/ using document root <none> and password file <none>
I20260619 16:56:43.868237 14387 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:43.868279 14387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:43.868417 14387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:56:43.869410 14387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "517c6fe86f124376a4d18fd6e8544ff9"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "66b1278ddac990fa09feec09553792c3"
server_key_iv: "cf9c90705493fff1575faa13ed783574"
server_key_version: "encryptionkey@0"
I20260619 16:56:43.869757 14387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "517c6fe86f124376a4d18fd6e8544ff9"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "66b1278ddac990fa09feec09553792c3"
server_key_iv: "cf9c90705493fff1575faa13ed783574"
server_key_version: "encryptionkey@0"
I20260619 16:56:43.871011 14387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:43.871707 14402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:43.871898 14387 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:56:43.871989 14387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "517c6fe86f124376a4d18fd6e8544ff9"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "66b1278ddac990fa09feec09553792c3"
server_key_iv: "cf9c90705493fff1575faa13ed783574"
server_key_version: "encryptionkey@0"
I20260619 16:56:43.872059 14387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:43.888247 14387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:43.890460 14387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:43.890617 14387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:43.890825 14387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:43.891142 14387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:43.891194 14387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:43.891227 14387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:43.891251 14387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:43.896801 14387 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:45427
I20260619 16:56:43.896885 14515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:45427 every 8 connection(s)
I20260619 16:56:43.897161 14387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:56:43.901399 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 14387
I20260619 16:56:43.901512 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:56:43.901705 31314 external_mini_cluster.cc:1428] Setting key 4c9b0da7f0e3bad023d4c6237f1db8e9
Jun 19 16:56:43 dist-test-slave-xb9n krb5kdc[14146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888203, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:43.906822 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:43.897825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:49325)
0619 16:56:43.898019 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:43.898072 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:43.898537 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:43.898787 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:43.898790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:43.899322 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:43.899500 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:43.899504 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:43.900188 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:43.900189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:43.900959 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:43.900960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:43.901336 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:43.901337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:43.901530 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:43.902046 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:43.902056 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:43.903588 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:43.905425 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:43.905431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:43.905435 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:43.905638 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:43.905873 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:43.905874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:43.905875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:43.906005 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:43.906296 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:43.906301 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:43.906409 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 16:56:43.906566 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 16:56:43.906679 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260619 16:56:43.906817 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:43.897841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:49325 (local address 127.30.148.190:36793)
0619 16:56:43.897886 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:43.897892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:43.898140 (+ 248us) server_negotiation.cc:407] Connection header received
0619 16:56:43.898648 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:43.898649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:43.898663 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:43.898718 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:43.899596 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:43.900093 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:43.901049 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:43.901241 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:43.901326 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:43.903697 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:43.903698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:43.903699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:43.903706 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:43.905300 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:43.905717 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:43.905718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:43.905719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:43.905780 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:43.906081 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:43.906082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:43.906082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:43.906226 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:43.906303 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:43.906610 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 16:56:43.906698 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:56:43.907286 14516 heartbeater.cc:347] Connected to a master server at 127.30.148.190:36793
I20260619 16:56:43.907402 14516 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:43.907617 14516 heartbeater.cc:511] Master 127.30.148.190:36793 requested a full tablet report, sending...
I20260619 16:56:43.908025 14191 ts_manager.cc:194] Registered new tserver with Master: 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427)
I20260619 16:56:43.908396 14191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:49325
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:56:43.925724 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36793
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:56:44.002986 14521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:56:44.003165 14521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:56:44.003185 14521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:56:44.004280 14521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:56:44.004319 14521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:56:44.004343 14521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:56:44.005710 14521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:56:44.005939 14521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:56:44.006146 14521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:56:44.010905 14526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:44.010934 14529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:56:44.010950 14527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:56:44.011056 14521 server_base.cc:1034] running on GCE node
I20260619 16:56:44.011268 14521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:56:44.011508 14521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:56:44.012682 14521 hybrid_clock.cc:648] HybridClock initialized: now 1781888204012661 us; error 33 us; skew 500 ppm
Jun 19 16:56:44 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888204, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:56:44.015039 14521 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:56:44.015416 14521 webserver.cc:460] Webserver started at http://127.30.148.131:34813/ using document root <none> and password file <none>
I20260619 16:56:44.015645 14521 fs_manager.cc:359] Metadata directory not provided
I20260619 16:56:44.015712 14521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:56:44.015856 14521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:56:44.016829 14521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "1fe2b12a7c8341c3a20d9c239c9ee163"
format_stamp: "Formatted at 2026-06-19 16:56:44 on dist-test-slave-xb9n"
server_key: "af66cc48e69ce4ce315bcf7146315c29"
server_key_iv: "1b6b5ae20f68d042440e26eea5dd37f8"
server_key_version: "encryptionkey@0"
I20260619 16:56:44.017175 14521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "1fe2b12a7c8341c3a20d9c239c9ee163"
format_stamp: "Formatted at 2026-06-19 16:56:44 on dist-test-slave-xb9n"
server_key: "af66cc48e69ce4ce315bcf7146315c29"
server_key_iv: "1b6b5ae20f68d042440e26eea5dd37f8"
server_key_version: "encryptionkey@0"
I20260619 16:56:44.018430 14521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:44.019078 14536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:44.019366 14521 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:56:44.019460 14521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "1fe2b12a7c8341c3a20d9c239c9ee163"
format_stamp: "Formatted at 2026-06-19 16:56:44 on dist-test-slave-xb9n"
server_key: "af66cc48e69ce4ce315bcf7146315c29"
server_key_iv: "1b6b5ae20f68d042440e26eea5dd37f8"
server_key_version: "encryptionkey@0"
I20260619 16:56:44.019541 14521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:56:44.030140 14521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:56:44.031803 14521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:56:44.031950 14521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:56:44.032157 14521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:56:44.032434 14521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:56:44.032485 14521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:44.032516 14521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:56:44.032541 14521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:56:44.038053 14521 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:45007
I20260619 16:56:44.038102 14649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:45007 every 8 connection(s)
I20260619 16:56:44.038375 14521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:56:44.039669 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 14521
I20260619 16:56:44.039773 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:56:44.040015 31314 external_mini_cluster.cc:1428] Setting key 854ce662ccb6cee41b71e55b6c1b7603
Jun 19 16:56:44 dist-test-slave-xb9n krb5kdc[14146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888204, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:56:44.047290 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.039002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:40609 (local address 127.30.148.190:36793)
0619 16:56:44.039084 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:44.039087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:44.039210 (+ 123us) server_negotiation.cc:407] Connection header received
0619 16:56:44.039704 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:44.039705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:44.039715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:44.039796 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:44.041036 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.041592 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.042473 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.042647 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.042697 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.044680 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:44.044681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:44.044681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:44.044686 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:44.045955 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:44.046545 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:44.046546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:44.046546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:44.046593 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:44.046803 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:44.046803 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:44.046804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:44.046897 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:44.046945 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:44.047174 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 16:56:44.047230 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 16:56:44.047441 14652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.038962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:40609)
0619 16:56:44.039106 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:44.039136 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:44.039606 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:44.039911 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:44.039916 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:44.040681 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:56:44.040936 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:44.040943 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.041664 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.041665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:44.042404 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:44.042406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.042714 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.042715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:44.042806 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.043265 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:56:44.043273 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:56:44.044608 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:56:44.046258 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:44.046264 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:44.046265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:44.046472 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:44.046652 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:56:44.046652 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:56:44.046652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:56:44.046741 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:56:44.046963 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:56:44.046967 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:56:44.047069 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 16:56:44.047149 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 16:56:44.047253 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":87,"threads_started":1}
I20260619 16:56:44.047910 14650 heartbeater.cc:347] Connected to a master server at 127.30.148.190:36793
I20260619 16:56:44.047993 14650 heartbeater.cc:464] Registering TS with master...
I20260619 16:56:44.048113 14650 heartbeater.cc:511] Master 127.30.148.190:36793 requested a full tablet report, sending...
I20260619 16:56:44.048446 14191 ts_manager.cc:194] Registered new tserver with Master: 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007)
I20260619 16:56:44.048802 14191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:40609
I20260619 16:56:44.051714 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:56:44.058594 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.052910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36972 (local address 127.30.148.190:36793)
0619 16:56:44.052972 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:44.052973 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:44.053084 (+ 111us) server_negotiation.cc:407] Connection header received
0619 16:56:44.053126 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:44.053127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:44.053135 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:44.053184 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:56:44.053743 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.054226 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.054995 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.055173 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.055242 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.056346 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:56:44.056347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:56:44.056347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:56:44.056352 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:56:44.057755 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:44.058052 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:44.058053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:44.058054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:44.058098 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:56:44.058237 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:56:44.058238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:56:44.058238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:56:44.058342 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:56:44.058414 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:44.058491 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 16:56:44.058543 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 16:56:44.060240 14191 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:56:44.061806 14191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:56:44.067981 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.063941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36209 (local address 127.0.0.1:38286)
0619 16:56:44.064135 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:44.064144 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:44.064185 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:44.064569 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:44.064570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:44.064572 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:44.064608 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:44.064609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.065621 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.065622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:44.066850 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:44.066853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.067734 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.067735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:44.067795 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.067799 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:56:44.067871 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 16:56:44.067909 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":1,"thread_start_us":53,"threads_started":1}
I20260619 16:56:44.068023 14663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.064158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38286 (local address 127.30.148.129:36209)
0619 16:56:44.064334 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:44.064337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:44.064345 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:56:44.064387 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:44.064389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:44.064402 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:44.064494 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:44.064758 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.065527 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.066940 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.067648 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.067738 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.067790 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:44.067885 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 16:56:44.067937 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":75,"threads_started":1}
I20260619 16:56:44.068109 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.063940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.30.148.130:45427)
0619 16:56:44.064304 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:44.064308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:44.064315 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:56:44.064345 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:44.064347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:44.064371 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:44.064464 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:44.064665 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.065769 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.067018 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.067487 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.067584 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.067639 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:44.067954 (+ 315us) server_negotiation.cc:299] Negotiation successful
0619 16:56:44.067997 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":70,"threads_started":1}
I20260619 16:56:44.068264 14660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.063842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58586 (local address 127.30.148.131:45007)
0619 16:56:44.063996 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:44.064001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:44.064538 (+ 537us) server_negotiation.cc:407] Connection header received
0619 16:56:44.064588 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:44.064591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:44.064605 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:44.064693 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:44.064882 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.065869 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.067045 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.067685 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.067744 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.067790 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:44.068141 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 16:56:44.068186 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260619 16:56:44.068078 14658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.063752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:45007 (local address 127.0.0.1:58586)
0619 16:56:44.064090 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:44.064102 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:44.064163 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:44.064760 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:44.064761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:44.064763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:44.064804 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:44.064806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.065968 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.065969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:44.066958 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:44.066961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.067825 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.067838 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:44.067902 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.067906 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:56:44.067957 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 16:56:44.067995 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":3,"spinlock_wait_cycles":768,"thread_start_us":51,"threads_started":1}
I20260619 16:56:44.067894 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.063752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:45427 (local address 127.0.0.1:32772)
0619 16:56:44.064035 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:44.064092 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:44.064160 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:44.064457 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:44.064460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:44.064462 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:44.064519 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:44.064521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.065867 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.065868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:44.066948 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:44.066950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.067576 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.067577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:44.067654 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.067667 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 16:56:44.067735 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 16:56:44.067786 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":1,"thread_start_us":53,"threads_started":1}
I20260619 16:56:44.069085 14584 tablet_service.cc:1467] Processing CreateTablet for tablet 494a15f5c49c4242bb28824b24627413 (DEFAULT_TABLE table=test-table [id=6cda7984d94247f4ac90ee9a9c154ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:56:44.069103 14316 tablet_service.cc:1467] Processing CreateTablet for tablet 494a15f5c49c4242bb28824b24627413 (DEFAULT_TABLE table=test-table [id=6cda7984d94247f4ac90ee9a9c154ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:56:44.069428 14584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494a15f5c49c4242bb28824b24627413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:44.069449 14316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494a15f5c49c4242bb28824b24627413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:44.069334 14450 tablet_service.cc:1467] Processing CreateTablet for tablet 494a15f5c49c4242bb28824b24627413 (DEFAULT_TABLE table=test-table [id=6cda7984d94247f4ac90ee9a9c154ef3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:56:44.069653 14450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494a15f5c49c4242bb28824b24627413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:56:44.071885 14664 tablet_bootstrap.cc:492] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163: Bootstrap starting.
I20260619 16:56:44.072667 14664 tablet_bootstrap.cc:654] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:44.072667 14665 tablet_bootstrap.cc:492] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Bootstrap starting.
I20260619 16:56:44.072904 14666 tablet_bootstrap.cc:492] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Bootstrap starting.
I20260619 16:56:44.073318 14664 log.cc:824] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:44.073333 14665 tablet_bootstrap.cc:654] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:44.073506 14666 tablet_bootstrap.cc:654] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Neither blocks nor log segments found. Creating new log.
I20260619 16:56:44.073907 14665 log.cc:824] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:44.074149 14664 tablet_bootstrap.cc:492] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163: No bootstrap required, opened a new log
I20260619 16:56:44.074150 14666 log.cc:824] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Log is configured to *not* fsync() on all Append() calls
I20260619 16:56:44.074220 14664 ts_tablet_manager.cc:1397] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 16:56:44.074609 14665 tablet_bootstrap.cc:492] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: No bootstrap required, opened a new log
I20260619 16:56:44.074676 14665 ts_tablet_manager.cc:1397] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:56:44.074834 14666 tablet_bootstrap.cc:492] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: No bootstrap required, opened a new log
I20260619 16:56:44.074913 14666 ts_tablet_manager.cc:1397] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 16:56:44.076333 14664 raft_consensus.cc:348] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:56:44.076524 14664 raft_consensus.cc:374] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:44.076560 14664 raft_consensus.cc:729] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe2b12a7c8341c3a20d9c239c9ee163, State: Initialized, Role: FOLLOWER
I20260619 16:56:44.076490 14666 raft_consensus.cc:348] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:56:44.076684 14666 raft_consensus.cc:374] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:44.076704 14664 consensus_queue.cc:260] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:56:44.076751 14666 raft_consensus.cc:729] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18aa84a3feda4c939c4b5282191eeef5, State: Initialized, Role: FOLLOWER
I20260619 16:56:44.076697 14665 raft_consensus.cc:348] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:56:44.076860 14665 raft_consensus.cc:374] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:56:44.076918 14665 raft_consensus.cc:729] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 517c6fe86f124376a4d18fd6e8544ff9, State: Initialized, Role: FOLLOWER
I20260619 16:56:44.076900 14666 consensus_queue.cc:260] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:56:44.076974 14664 ts_tablet_manager.cc:1428] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 16:56:44.077032 14665 consensus_queue.cc:260] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:56:44.077131 14650 heartbeater.cc:503] Master 127.30.148.190:36793 was elected leader, sending a full tablet report...
I20260619 16:56:44.077183 14382 heartbeater.cc:503] Master 127.30.148.190:36793 was elected leader, sending a full tablet report...
I20260619 16:56:44.077296 14665 ts_tablet_manager.cc:1428] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:56:44.077407 14516 heartbeater.cc:503] Master 127.30.148.190:36793 was elected leader, sending a full tablet report...
I20260619 16:56:44.077183 14666 ts_tablet_manager.cc:1428] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260619 16:56:44.147725 14517 tablet.cc:2369] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:56:44.233695 14672 raft_consensus.cc:482] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:56:44.233801 14672 raft_consensus.cc:504] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:56:44.234081 14672 leader_election.cc:288] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007), 18aa84a3feda4c939c4b5282191eeef5 (127.30.148.129:36209)
I20260619 16:56:44.236737 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.234249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:45007 (local address 127.30.148.130:35839)
0619 16:56:44.234308 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:44.234320 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:44.234400 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:44.234533 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:44.234534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:44.234536 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:44.234611 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:44.234612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.235217 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.235218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:44.235924 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:44.235926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.236483 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.236484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:44.236563 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.236566 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:56:44.236632 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 16:56:44.236670 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 16:56:44.236737 14660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.234355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:35839 (local address 127.30.148.131:45007)
0619 16:56:44.234404 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:44.234406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:44.234439 (+ 33us) server_negotiation.cc:407] Connection header received
0619 16:56:44.234468 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:44.234469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:44.234478 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:44.234530 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:44.234682 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.235135 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.236019 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.236389 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.236484 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.236493 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:44.236637 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 16:56:44.236673 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 16:56:44.236848 14663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.234439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:50389 (local address 127.30.148.129:36209)
0619 16:56:44.234487 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:44.234489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:44.234568 (+ 79us) server_negotiation.cc:407] Connection header received
0619 16:56:44.234617 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:44.234618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:44.234626 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:44.234693 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:56:44.234851 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.235291 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.236169 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.236513 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.236595 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.236603 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:44.236748 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 16:56:44.236779 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 16:56:44.236860 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.234358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36209 (local address 127.30.148.130:50389)
0619 16:56:44.234502 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:44.234511 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:56:44.234558 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:56:44.234697 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:56:44.234698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:56:44.234699 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:56:44.234773 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:44.234776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.235383 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.235384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:44.236076 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:56:44.236078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.236595 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.236596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:56:44.236664 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.236667 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:56:44.236741 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 16:56:44.236778 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260619 16:56:44.237236 14604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494a15f5c49c4242bb28824b24627413" candidate_uuid: "517c6fe86f124376a4d18fd6e8544ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" is_pre_election: true
I20260619 16:56:44.237233 14336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494a15f5c49c4242bb28824b24627413" candidate_uuid: "517c6fe86f124376a4d18fd6e8544ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18aa84a3feda4c939c4b5282191eeef5" is_pre_election: true
I20260619 16:56:44.237422 14336 raft_consensus.cc:2454] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 517c6fe86f124376a4d18fd6e8544ff9 in term 0.
I20260619 16:56:44.237422 14604 raft_consensus.cc:2454] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 517c6fe86f124376a4d18fd6e8544ff9 in term 0.
I20260619 16:56:44.237671 14405 leader_election.cc:302] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18aa84a3feda4c939c4b5282191eeef5, 517c6fe86f124376a4d18fd6e8544ff9; no voters:
I20260619 16:56:44.237792 14672 raft_consensus.cc:2781] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:56:44.237844 14672 raft_consensus.cc:482] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:56:44.237869 14672 raft_consensus.cc:3037] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:44.238509 14672 raft_consensus.cc:504] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:56:44.238626 14672 leader_election.cc:288] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007), 18aa84a3feda4c939c4b5282191eeef5 (127.30.148.129:36209)
I20260619 16:56:44.238829 14336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494a15f5c49c4242bb28824b24627413" candidate_uuid: "517c6fe86f124376a4d18fd6e8544ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18aa84a3feda4c939c4b5282191eeef5"
I20260619 16:56:44.238854 14604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494a15f5c49c4242bb28824b24627413" candidate_uuid: "517c6fe86f124376a4d18fd6e8544ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163"
I20260619 16:56:44.238906 14336 raft_consensus.cc:3037] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:44.238940 14604 raft_consensus.cc:3037] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:56:44.239557 14336 raft_consensus.cc:2454] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 517c6fe86f124376a4d18fd6e8544ff9 in term 1.
I20260619 16:56:44.239655 14604 raft_consensus.cc:2454] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 517c6fe86f124376a4d18fd6e8544ff9 in term 1.
I20260619 16:56:44.239714 14405 leader_election.cc:302] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18aa84a3feda4c939c4b5282191eeef5, 517c6fe86f124376a4d18fd6e8544ff9; no voters:
I20260619 16:56:44.239810 14672 raft_consensus.cc:2781] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:56:44.239969 14672 raft_consensus.cc:686] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 1 LEADER]: Becoming Leader. State: Replica: 517c6fe86f124376a4d18fd6e8544ff9, State: Running, Role: LEADER
I20260619 16:56:44.240103 14672 consensus_queue.cc:237] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:56:44.240916 14190 catalog_manager.cc:5495] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130). New cstate: current_term: 1 leader_uuid: "517c6fe86f124376a4d18fd6e8544ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } health_report { overall_health: HEALTHY } } }
I20260619 16:56:44.247467 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.245527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32776 (local address 127.30.148.130:45427)
0619 16:56:44.245578 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 16:56:44.245579 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:56:44.245619 (+ 40us) server_negotiation.cc:407] Connection header received
0619 16:56:44.245688 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:56:44.245689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:56:44.245702 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:56:44.245752 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:56:44.245861 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.246281 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.246949 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:56:44.247108 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:56:44.247155 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:56:44.247275 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:56:44.247349 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:56:44.247400 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:56:44.247412 (+ 12us) server_negotiation.cc:299] Negotiation successful
0619 16:56:44.247425 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 16:56:44.247695 14383 tablet.cc:2369] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:56:44.249123 14604 raft_consensus.cc:1261] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 1 FOLLOWER]: Refusing update from remote peer 517c6fe86f124376a4d18fd6e8544ff9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:56:44.249123 14336 raft_consensus.cc:1261] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Refusing update from remote peer 517c6fe86f124376a4d18fd6e8544ff9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:56:44.249470 14674 consensus_queue.cc:1035] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:56:44.249590 14672 consensus_queue.cc:1035] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:56:44.251606 14677 mvcc.cc:205] Tried to move back new op lower bound from 7298614084601802752 to 7298614084568002560. Current Snapshot: MvccSnapshot[applied={T|T < 7298614084601802752}]
I20260619 16:56:44.254400 14191 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:56:44.254489 14191 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:56:44.255209 14191 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Sending DeleteTablet for 3 replicas of tablet 494a15f5c49c4242bb28824b24627413
I20260619 16:56:44.255527 14584 tablet_service.cc:1514] Processing DeleteTablet for tablet 494a15f5c49c4242bb28824b24627413 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:44 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:58586
I20260619 16:56:44.255599 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 14253
I20260619 16:56:44.255573 14316 tablet_service.cc:1514] Processing DeleteTablet for tablet 494a15f5c49c4242bb28824b24627413 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:44 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:38286
I20260619 16:56:44.255633 14450 tablet_service.cc:1514] Processing DeleteTablet for tablet 494a15f5c49c4242bb28824b24627413 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:44 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:32772
I20260619 16:56:44.258764 14687 tablet_replica.cc:330] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163: stopping tablet replica
I20260619 16:56:44.258910 14687 raft_consensus.cc:2229] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:56:44.259050 14687 raft_consensus.cc:2258] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:56:44.259500 14687 ts_tablet_manager.cc:1905] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:56:44.260958 14687 ts_tablet_manager.cc:1918] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:56:44.261044 14687 log.cc:1196] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/494a15f5c49c4242bb28824b24627413
I20260619 16:56:44.261303 14687 ts_tablet_manager.cc:1939] T 494a15f5c49c4242bb28824b24627413 P 1fe2b12a7c8341c3a20d9c239c9ee163: Deleting consensus metadata
I20260619 16:56:44.261704 14177 catalog_manager.cc:4841] TS 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007): tablet 494a15f5c49c4242bb28824b24627413 (table test-table [id=6cda7984d94247f4ac90ee9a9c154ef3]) successfully deleted
W20260619 16:56:44.262897 14177 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:36209 (error 108)
I20260619 16:56:44.263055 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 14387
I20260619 16:56:44.265980 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.265856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36209 (local address 127.0.0.1:38288)
0619 16:56:44.265921 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:44.265955 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 16:56:44.266091 14177 catalog_manager.cc:4584] TS 18aa84a3feda4c939c4b5282191eeef5 (127.30.148.129:36209): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 494a15f5c49c4242bb28824b24627413: Network error: Client connection negotiation failed: client connection to 127.30.148.129:36209: connect: Connection refused (error 111)
I20260619 16:56:44.267400 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:56:44.267310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:45427 (local address 127.0.0.1:32780)
0619 16:56:44.267355 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 16:56:44.267378 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.130:45427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 16:56:44.267762 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 14521
I20260619 16:56:44.272573 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 14159
I20260619 16:57:04.278630 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36793
--webserver_interface=127.30.148.190
--webserver_port=42921
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:36793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:04.358839 14691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:04.359019 14691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:04.359046 14691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:04.360172 14691 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 16:57:04.360208 14691 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 16:57:04.360220 14691 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 16:57:04.360258 14691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:04.360288 14691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:04.360306 14691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:04.360332 14691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:04.360381 14691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:04.361943 14691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:36793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=42921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:04.362179 14691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:04.362412 14691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:04.370956 14698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:04.370958 14697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:04.370992 14691 server_base.cc:1034] running on GCE node
W20260619 16:57:04.370991 14700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:04.371479 14691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:04.371763 14691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:04.372900 14691 hybrid_clock.cc:648] HybridClock initialized: now 1781888224372887 us; error 27 us; skew 500 ppm
Jun 19 16:57:04 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888224, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:04.383800 14691 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:57:04.384272 14691 webserver.cc:460] Webserver started at http://127.30.148.190:42921/ using document root <none> and password file <none>
I20260619 16:57:04.384486 14691 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:04.384541 14691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:04.385951 14691 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:04.386624 14707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:04.386808 14691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 16:57:04.386885 14691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "d97b84b43b26421eb17dc43b5df1ecbc"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "a2f1ea3f9319263b6e6152d3e2ca4094"
server_key_iv: "fb009932bd70e217cc9e636919a3558b"
server_key_version: "encryptionkey@0"
I20260619 16:57:04.387266 14691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:04.397773 14691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:04.399503 14691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:04.399606 14691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:04.403650 14691 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:36793
I20260619 16:57:04.403728 14759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:36793 every 8 connection(s)
I20260619 16:57:04.404083 14691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:04.405490 14760 sys_catalog.cc:263] Verifying existing consensus state
I20260619 16:57:04.406147 14760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Bootstrap starting.
I20260619 16:57:04.413704 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 14691
I20260619 16:57:04.414001 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:36209
--local_ip_for_outbound_sockets=127.30.148.129
--tserver_master_addrs=127.30.148.190:36793
--webserver_port=45717
--webserver_interface=127.30.148.129
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:57:04.414623 14760 log.cc:824] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:04.417141 14760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:57:04.417500 14760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Bootstrap complete.
I20260619 16:57:04.421536 14760 raft_consensus.cc:348] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } }
I20260619 16:57:04.421911 14760 raft_consensus.cc:729] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97b84b43b26421eb17dc43b5df1ecbc, State: Initialized, Role: FOLLOWER
I20260619 16:57:04.422091 14760 consensus_queue.cc:260] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } }
I20260619 16:57:04.422180 14760 raft_consensus.cc:388] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:04.422241 14760 raft_consensus.cc:482] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:04.422298 14760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 1 FOLLOWER]: Advancing to term 2
I20260619 16:57:04.423877 14760 raft_consensus.cc:504] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } }
I20260619 16:57:04.423995 14760 leader_election.cc:302] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d97b84b43b26421eb17dc43b5df1ecbc; no voters:
I20260619 16:57:04.424207 14760 leader_election.cc:288] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 16:57:04.424276 14764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 2 FOLLOWER]: Leader election won for term 2
I20260619 16:57:04.424460 14764 raft_consensus.cc:686] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 2 LEADER]: Becoming Leader. State: Replica: d97b84b43b26421eb17dc43b5df1ecbc, State: Running, Role: LEADER
I20260619 16:57:04.424521 14760 sys_catalog.cc:564] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:04.424590 14764 consensus_queue.cc:237] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } }
I20260619 16:57:04.424985 14765 sys_catalog.cc:455] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } } }
I20260619 16:57:04.425064 14765 sys_catalog.cc:458] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:04.425324 14766 sys_catalog.cc:455] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d97b84b43b26421eb17dc43b5df1ecbc. Latest consensus state: current_term: 2 leader_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } } }
I20260619 16:57:04.425406 14766 sys_catalog.cc:458] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:04.425588 14769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:04.426352 14769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:04.426600 14769 catalog_manager.cc:1259] Loaded cluster ID: c9895b0ec9e94ddb8b834f839110174b
I20260619 16:57:04.426687 14769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:04.427807 14769 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:04.428009 14769 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Loaded TSK: 0
I20260619 16:57:04.437991 14769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Generated new TSK 1
I20260619 16:57:04.438077 14769 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 16:57:04.497514 14763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:04.497694 14763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:04.497715 14763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:04.498944 14763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:04.498986 14763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:04.499011 14763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:57:04.500334 14763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:36209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=45717
--tserver_master_addrs=127.30.148.190:36793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:04.500555 14763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:04.500793 14763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:04.505501 14789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:04.505556 14788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:04.505852 14791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:04.505863 14763 server_base.cc:1034] running on GCE node
I20260619 16:57:04.506108 14763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:04.506377 14763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:04.507571 14763 hybrid_clock.cc:648] HybridClock initialized: now 1781888224507551 us; error 31 us; skew 500 ppm
Jun 19 16:57:04 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888224, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:04.510376 14763 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:57:04.510807 14763 webserver.cc:460] Webserver started at http://127.30.148.129:45717/ using document root <none> and password file <none>
I20260619 16:57:04.511034 14763 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:04.511087 14763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:04.512398 14763 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:04.513067 14798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:04.513218 14763 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:04.513317 14763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "18aa84a3feda4c939c4b5282191eeef5"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "50bc5059e85298507082ba4714333fd1"
server_key_iv: "1278d3db870df7718b0e9665921080f5"
server_key_version: "encryptionkey@0"
I20260619 16:57:04.513681 14763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:04.529743 14763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:04.532020 14763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:04.532159 14763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:04.532421 14763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:04.532892 14805 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 16:57:04.534353 14763 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 16:57:04.534407 14763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 16:57:04.534432 14763 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 16:57:04.535073 14763 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 16:57:04.535115 14763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 16:57:04.535158 14805 tablet_bootstrap.cc:492] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Bootstrap starting.
I20260619 16:57:04.542806 14763 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:36209
I20260619 16:57:04.543350 14763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:57:04.543923 14912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:36209 every 8 connection(s)
I20260619 16:57:04.546900 14805 log.cc:824] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:04.549199 14805 tablet_bootstrap.cc:492] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:57:04.549575 14805 tablet_bootstrap.cc:492] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Bootstrap complete.
I20260619 16:57:04.549559 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 14763
I20260619 16:57:04.549836 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:45427
--local_ip_for_outbound_sockets=127.30.148.130
--tserver_master_addrs=127.30.148.190:36793
--webserver_port=35793
--webserver_interface=127.30.148.130
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:57:04.551119 14805 ts_tablet_manager.cc:1397] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.010s
Jun 19 16:57:04 dist-test-slave-xb9n krb5kdc[14146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888224, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:04.554421 14805 raft_consensus.cc:348] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:57:04.554913 14805 raft_consensus.cc:729] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18aa84a3feda4c939c4b5282191eeef5, State: Initialized, Role: FOLLOWER
I20260619 16:57:04.555132 14805 consensus_queue.cc:260] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:57:04.555503 14805 ts_tablet_manager.cc:1428] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 16:57:04.557077 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.544234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:57877 (local address 127.30.148.190:36793)
0619 16:57:04.545486 (+ 1252us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:04.545499 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:04.545554 (+ 55us) server_negotiation.cc:407] Connection header received
0619 16:57:04.545698 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:04.545705 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:04.545766 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:04.545928 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:04.546852 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.548144 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.549674 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.550253 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.550785 (+ 532us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.553163 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:04.553167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:04.553167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:04.553178 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:04.555144 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:04.555608 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:04.555611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:04.555611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:04.555711 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:04.556035 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:04.556035 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:04.556036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:04.556314 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:04.556403 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:04.556801 (+ 398us) server_negotiation.cc:299] Negotiation successful
0619 16:57:04.556941 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1196,"thread_start_us":101,"threads_started":1}
I20260619 16:57:04.557153 14915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.544156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:57877)
0619 16:57:04.544374 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:04.544430 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:04.545177 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:04.545966 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:04.545971 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:04.546347 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:04.546689 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.546718 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.548281 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.548282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.549577 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.549580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.550371 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.550372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.550512 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.551109 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:04.551118 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:04.553025 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:04.555237 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:04.555243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:04.555245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:04.555517 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:04.555796 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:04.555797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:04.555798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:04.555946 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:04.556404 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:04.556411 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:04.556592 (+ 181us) client_negotiation.cc:769] Sending connection context
0619 16:57:04.556723 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 16:57:04.556907 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":98,"threads_started":1}
I20260619 16:57:04.557893 14913 heartbeater.cc:347] Connected to a master server at 127.30.148.190:36793
I20260619 16:57:04.558020 14913 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:04.558180 14913 heartbeater.cc:511] Master 127.30.148.190:36793 requested a full tablet report, sending...
I20260619 16:57:04.558869 14724 ts_manager.cc:194] Registered new tserver with Master: 18aa84a3feda4c939c4b5282191eeef5 (127.30.148.129:36209)
I20260619 16:57:04.559912 14724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:57877
I20260619 16:57:04.560694 14913 heartbeater.cc:503] Master 127.30.148.190:36793 was elected leader, sending a full tablet report...
Jun 19 16:57:04 dist-test-slave-xb9n krb5kdc[14146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888224, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.129@KRBTEST.COM
I20260619 16:57:04.568321 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.559523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44712 (local address 127.30.148.129:36209)
0619 16:57:04.559885 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:04.559890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:04.559898 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:57:04.559940 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:04.559942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:04.559955 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:04.560019 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:04.561074 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.562277 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.562970 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.563400 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.563474 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.565581 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:04.565588 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:04.565588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:04.565594 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:04.567076 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:04.567503 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:04.567505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:04.567505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:04.567553 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:04.567781 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:04.567781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:04.567782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:04.567900 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:04.567990 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:04.568193 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 16:57:04.568262 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":70,"threads_started":1}
I20260619 16:57:04.568373 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.559449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36209 (local address 127.0.0.1:44712)
0619 16:57:04.559685 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:04.559719 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:04.559779 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:04.560077 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:04.560081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:04.560905 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:04.560988 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.560990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.562350 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.562350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.562885 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.562887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.563471 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.563472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.563522 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.563976 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:04.563980 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:04.565506 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:04.567155 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:04.567161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:04.567163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:04.567422 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:04.567642 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:04.567642 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:04.567643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:04.567715 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:04.567999 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:04.568006 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:04.568062 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 16:57:04.568161 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 16:57:04.568274 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":82,"threads_started":1}
I20260619 16:57:04.568717 14830 tablet_service.cc:1514] Processing DeleteTablet for tablet 494a15f5c49c4242bb28824b24627413 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:44 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:44712
I20260619 16:57:04.569231 14922 tablet_replica.cc:330] stopping tablet replica
I20260619 16:57:04.569335 14922 raft_consensus.cc:2229] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:04.569455 14922 raft_consensus.cc:2258] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:04.569753 14922 ts_tablet_manager.cc:1905] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:04.572062 14922 ts_tablet_manager.cc:1918] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:04.572135 14922 log.cc:1196] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal/wals/494a15f5c49c4242bb28824b24627413
I20260619 16:57:04.572351 14922 ts_tablet_manager.cc:1939] T 494a15f5c49c4242bb28824b24627413 P 18aa84a3feda4c939c4b5282191eeef5: Deleting consensus metadata
I20260619 16:57:04.572811 14710 catalog_manager.cc:4841] TS 18aa84a3feda4c939c4b5282191eeef5 (127.30.148.129:36209): tablet 494a15f5c49c4242bb28824b24627413 (table test-table [id=6cda7984d94247f4ac90ee9a9c154ef3]) successfully deleted
W20260619 16:57:04.635008 14918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:04.635192 14918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:04.635215 14918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:04.636272 14918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:04.636303 14918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:04.636328 14918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:57:04.637781 14918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:45427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=35793
--tserver_master_addrs=127.30.148.190:36793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:04.638132 14918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:04.638360 14918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:04.644783 14928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:04.644860 14930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:04.645140 14918 server_base.cc:1034] running on GCE node
W20260619 16:57:04.645146 14927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:04.645453 14918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:04.645705 14918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:04.646927 14918 hybrid_clock.cc:648] HybridClock initialized: now 1781888224646910 us; error 32 us; skew 500 ppm
Jun 19 16:57:04 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888224, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:04.649684 14918 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:57:04.650108 14918 webserver.cc:460] Webserver started at http://127.30.148.130:35793/ using document root <none> and password file <none>
I20260619 16:57:04.650336 14918 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:04.650418 14918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:04.651727 14918 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:04.652384 14937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:04.652537 14918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:04.652631 14918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "517c6fe86f124376a4d18fd6e8544ff9"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "66b1278ddac990fa09feec09553792c3"
server_key_iv: "cf9c90705493fff1575faa13ed783574"
server_key_version: "encryptionkey@0"
I20260619 16:57:04.652956 14918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:04.669282 14918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:04.671590 14918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:04.671751 14918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:04.671967 14918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:04.672356 14944 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 16:57:04.673761 14918 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 16:57:04.673817 14918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 16:57:04.673862 14918 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 16:57:04.674468 14918 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 16:57:04.674554 14918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 16:57:04.674611 14944 tablet_bootstrap.cc:492] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Bootstrap starting.
I20260619 16:57:04.680894 14918 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:45427
I20260619 16:57:04.680974 15051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:45427 every 8 connection(s)
I20260619 16:57:04.681257 14918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:57:04.684422 14944 log.cc:824] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:04.686152 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 14918
I20260619 16:57:04.686427 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:45007
--local_ip_for_outbound_sockets=127.30.148.131
--tserver_master_addrs=127.30.148.190:36793
--webserver_port=34813
--webserver_interface=127.30.148.131
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:57:04.686883 14944 tablet_bootstrap.cc:492] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
Jun 19 16:57:04 dist-test-slave-xb9n krb5kdc[14146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888224, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:04.687197 14944 tablet_bootstrap.cc:492] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Bootstrap complete.
I20260619 16:57:04.688310 14944 ts_tablet_manager.cc:1397] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260619 16:57:04.689402 14944 raft_consensus.cc:348] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:57:04.689687 14944 raft_consensus.cc:729] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 517c6fe86f124376a4d18fd6e8544ff9, State: Initialized, Role: FOLLOWER
I20260619 16:57:04.689833 14944 consensus_queue.cc:260] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } }
I20260619 16:57:04.690168 14944 ts_tablet_manager.cc:1428] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260619 16:57:04.690894 15054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.681943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:35057)
0619 16:57:04.682119 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:04.682152 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:04.682604 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:04.682841 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:04.682845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:04.683109 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:04.683277 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.683281 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.684007 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.684008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.684860 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.684867 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.685309 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.685310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.685490 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.686004 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:04.686011 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:04.687545 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:04.689403 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:04.689410 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:04.689412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:04.689658 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:04.689957 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:04.689957 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:04.689958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:04.690130 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:04.690414 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:04.690418 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:04.690526 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 16:57:04.690609 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 16:57:04.690720 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
I20260619 16:57:04.690900 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.681992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:35057 (local address 127.30.148.190:36793)
0619 16:57:04.682036 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:04.682038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:04.682234 (+ 196us) server_negotiation.cc:407] Connection header received
0619 16:57:04.682716 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:04.682718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:04.682730 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:04.682776 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:04.683385 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.683892 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.684976 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.685191 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.685277 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.687642 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:04.687644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:04.687644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:04.687652 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:04.689276 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:04.689779 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:04.689780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:04.689780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:04.689849 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:04.690188 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:04.690189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:04.690189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:04.690342 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:04.690416 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:04.690656 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 16:57:04.690762 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:57:04.691350 15052 heartbeater.cc:347] Connected to a master server at 127.30.148.190:36793
I20260619 16:57:04.691459 15052 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:04.691612 15052 heartbeater.cc:511] Master 127.30.148.190:36793 requested a full tablet report, sending...
I20260619 16:57:04.692179 14724 ts_manager.cc:194] Registered new tserver with Master: 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427)
I20260619 16:57:04.692827 14724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:35057
I20260619 16:57:04.693887 15052 heartbeater.cc:503] Master 127.30.148.190:36793 was elected leader, sending a full tablet report...
Jun 19 16:57:04 dist-test-slave-xb9n krb5kdc[14146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888224, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.130@KRBTEST.COM
I20260619 16:57:04.700204 15058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.693029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.30.148.130:45427)
0619 16:57:04.693189 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:04.693192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:04.693200 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:57:04.693225 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:04.693226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:04.693238 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:04.693335 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:04.693638 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.694479 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.695147 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.695488 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.695564 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.697678 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:04.697684 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:04.697684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:04.697691 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:04.699146 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:04.699533 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:04.699535 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:04.699535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:04.699584 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:04.699770 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:04.699770 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:04.699770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:04.699884 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:04.699947 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:04.700082 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 16:57:04.700148 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260619 16:57:04.700249 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.692494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:45427 (local address 127.0.0.1:44200)
0619 16:57:04.692543 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:04.692552 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:04.692593 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:04.693337 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:04.693338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:04.693520 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:04.693555 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.693557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.694552 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.694553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.695057 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.695058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.695562 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.695563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.695629 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.696061 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:04.696065 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:04.697573 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:04.699231 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:04.699232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:04.699232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:04.699443 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:04.699648 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:04.699649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:04.699649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:04.699712 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:04.699944 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:04.699945 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:04.699987 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 16:57:04.700060 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 16:57:04.700153 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 16:57:04.700616 14986 tablet_service.cc:1514] Processing DeleteTablet for tablet 494a15f5c49c4242bb28824b24627413 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:56:44 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:44200
I20260619 16:57:04.701135 15059 tablet_replica.cc:330] stopping tablet replica
I20260619 16:57:04.701215 15059 raft_consensus.cc:2229] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:04.701297 15059 raft_consensus.cc:2258] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:04.701586 15059 ts_tablet_manager.cc:1905] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:04.703327 15059 ts_tablet_manager.cc:1918] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:04.703383 15059 log.cc:1196] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/494a15f5c49c4242bb28824b24627413
I20260619 16:57:04.703617 15059 ts_tablet_manager.cc:1939] T 494a15f5c49c4242bb28824b24627413 P 517c6fe86f124376a4d18fd6e8544ff9: Deleting consensus metadata
I20260619 16:57:04.704107 14708 catalog_manager.cc:4841] TS 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427): tablet 494a15f5c49c4242bb28824b24627413 (table test-table [id=6cda7984d94247f4ac90ee9a9c154ef3]) successfully deleted
W20260619 16:57:04.770884 15056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:04.771077 15056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:04.771099 15056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:04.772159 15056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:04.772192 15056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:04.772217 15056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:57:04.773694 15056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:45007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=34813
--tserver_master_addrs=127.30.148.190:36793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:04.773862 15056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:04.774029 15056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:04.778632 15064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:04.778632 15067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:04.778702 15065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:04.778836 15056 server_base.cc:1034] running on GCE node
I20260619 16:57:04.779122 15056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:04.779335 15056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:04.780488 15056 hybrid_clock.cc:648] HybridClock initialized: now 1781888224780475 us; error 24 us; skew 500 ppm
Jun 19 16:57:04 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888224, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:04.783043 15056 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:57:04.783474 15056 webserver.cc:460] Webserver started at http://127.30.148.131:34813/ using document root <none> and password file <none>
I20260619 16:57:04.783646 15056 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:04.783695 15056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:04.784902 15056 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:04.785533 15074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:04.785682 15056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 16:57:04.785764 15056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "1fe2b12a7c8341c3a20d9c239c9ee163"
format_stamp: "Formatted at 2026-06-19 16:56:44 on dist-test-slave-xb9n"
server_key: "af66cc48e69ce4ce315bcf7146315c29"
server_key_iv: "1b6b5ae20f68d042440e26eea5dd37f8"
server_key_version: "encryptionkey@0"
I20260619 16:57:04.786054 15056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:04.801889 15056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:04.803586 15056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:04.803690 15056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:04.803864 15056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:04.804087 15056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:04.804113 15056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:04.804152 15056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:04.804163 15056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:04.809556 15056 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:45007
I20260619 16:57:04.809643 15187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:45007 every 8 connection(s)
I20260619 16:57:04.809952 15056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:57:04.813434 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 15056
Jun 19 16:57:04 dist-test-slave-xb9n krb5kdc[14146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888224, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:04.819424 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.810721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:52085 (local address 127.30.148.190:36793)
0619 16:57:04.810770 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:04.810773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:04.810941 (+ 168us) server_negotiation.cc:407] Connection header received
0619 16:57:04.811431 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:04.811432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:04.811445 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:04.811498 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:04.812386 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.812875 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.814059 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.814250 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.814309 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.816502 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:04.816504 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:04.816504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:04.816511 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:04.818041 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:04.818462 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:04.818463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:04.818463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:04.818525 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:04.818772 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:04.818773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:04.818773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:04.818883 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:04.818956 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:04.819220 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 16:57:04.819338 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:57:04.819566 15190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.810681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:52085)
0619 16:57:04.810818 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:04.810860 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:04.811329 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:04.811569 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:04.811573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:04.812112 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:04.812287 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.812291 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.812968 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.812969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.813979 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.813983 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.814324 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.814325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.814451 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.814928 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:04.814938 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:04.816381 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:04.818137 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:04.818144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:04.818146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:04.818373 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:04.818594 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:04.818594 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:04.818595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:04.818702 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:04.818949 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:04.818954 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:04.819093 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 16:57:04.819199 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 16:57:04.819349 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260619 16:57:04.820164 15188 heartbeater.cc:347] Connected to a master server at 127.30.148.190:36793
I20260619 16:57:04.820255 15188 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:04.820416 15188 heartbeater.cc:511] Master 127.30.148.190:36793 requested a full tablet report, sending...
I20260619 16:57:04.820501 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.813814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58458 (local address 127.30.148.190:36793)
0619 16:57:04.813924 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:04.813926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:04.814339 (+ 413us) server_negotiation.cc:407] Connection header received
0619 16:57:04.814352 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:04.814353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:04.814360 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:04.814404 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:04.815186 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.815638 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.816442 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.816705 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.816788 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.817891 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:04.817892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:04.817893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:04.817899 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:04.819405 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:04.819771 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:04.819772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:04.819772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:04.819832 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:04.820015 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:04.820016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:04.820016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:04.820137 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:04.820286 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:04.820325 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 16:57:04.820396 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
I20260619 16:57:04.820973 14724 ts_manager.cc:194] Registered new tserver with Master: 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007)
I20260619 16:57:04.821203 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:57:04.821458 14724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:52085
I20260619 16:57:04.823704 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.821593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.30.148.190:36793)
0619 16:57:04.821625 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:04.821627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:04.821674 (+ 47us) server_negotiation.cc:407] Connection header received
0619 16:57:04.821741 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:04.821741 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:04.821770 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:04.821840 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:57:04.821955 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.822376 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.823070 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.823251 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.823302 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.823482 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:57:04.823556 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:57:04.823610 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:04.823650 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 16:57:04.823666 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 16:57:04.823972 14724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:57:04.825747 14724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:57:04.827811 14830 tablet_service.cc:1467] Processing CreateTablet for tablet 03c9eee2c552478b8ae2c4ccddd12c6a (DEFAULT_TABLE table=test-table [id=77d95b0411d842788ea9366ed23241d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:04.827814 14986 tablet_service.cc:1467] Processing CreateTablet for tablet 03c9eee2c552478b8ae2c4ccddd12c6a (DEFAULT_TABLE table=test-table [id=77d95b0411d842788ea9366ed23241d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:04.828118 14986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c9eee2c552478b8ae2c4ccddd12c6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:04.829066 14830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c9eee2c552478b8ae2c4ccddd12c6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:04.829702 14944 tablet_bootstrap.cc:492] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9: Bootstrap starting.
I20260619 16:57:04.829991 14944 tablet_bootstrap.cc:654] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:04.830598 14805 tablet_bootstrap.cc:492] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5: Bootstrap starting.
I20260619 16:57:04.830768 14944 tablet_bootstrap.cc:492] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9: No bootstrap required, opened a new log
I20260619 16:57:04.830822 14944 ts_tablet_manager.cc:1397] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:04.830911 14805 tablet_bootstrap.cc:654] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:04.830986 14944 raft_consensus.cc:348] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:04.831066 14944 raft_consensus.cc:374] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:04.831125 14944 raft_consensus.cc:729] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 517c6fe86f124376a4d18fd6e8544ff9, State: Initialized, Role: FOLLOWER
I20260619 16:57:04.831187 14944 consensus_queue.cc:260] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:04.831301 14944 ts_tablet_manager.cc:1428] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:04.831553 14805 tablet_bootstrap.cc:492] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5: No bootstrap required, opened a new log
I20260619 16:57:04.831602 14805 ts_tablet_manager.cc:1397] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 16:57:04.831724 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.827486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:45007 (local address 127.0.0.1:37388)
0619 16:57:04.827535 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:04.827543 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:04.827568 (+ 25us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:04.828047 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:04.828048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:04.828049 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:04.828087 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.828088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.829375 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.829376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.830427 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.830430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.831495 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.831496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.831560 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.831582 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 16:57:04.831614 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 16:57:04.831654 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 16:57:04.831961 14805 raft_consensus.cc:348] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:04.832041 14805 raft_consensus.cc:374] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:04.832120 14805 raft_consensus.cc:729] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18aa84a3feda4c939c4b5282191eeef5, State: Initialized, Role: FOLLOWER
I20260619 16:57:04.832166 14805 consensus_queue.cc:260] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:04.832173 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.827669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.30.148.131:45007)
0619 16:57:04.827832 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:04.827837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:04.827847 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:57:04.827882 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:04.827885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:04.827899 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:04.828161 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:04.828234 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.829272 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.830527 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.831402 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.831944 (+ 542us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.832009 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:04.832037 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 16:57:04.832090 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260619 16:57:04.832255 14805 ts_tablet_manager.cc:1428] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 16:57:04.832762 15122 tablet_service.cc:1467] Processing CreateTablet for tablet 03c9eee2c552478b8ae2c4ccddd12c6a (DEFAULT_TABLE table=test-table [id=77d95b0411d842788ea9366ed23241d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:04.833067 15122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c9eee2c552478b8ae2c4ccddd12c6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:04.835784 15197 tablet_bootstrap.cc:492] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163: Bootstrap starting.
I20260619 16:57:04.836371 15197 tablet_bootstrap.cc:654] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:04.836956 15197 log.cc:824] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:04.837637 15197 tablet_bootstrap.cc:492] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163: No bootstrap required, opened a new log
I20260619 16:57:04.837692 15197 ts_tablet_manager.cc:1397] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:57:04.839043 15197 raft_consensus.cc:348] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:04.839176 15197 raft_consensus.cc:374] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:04.839197 15197 raft_consensus.cc:729] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe2b12a7c8341c3a20d9c239c9ee163, State: Initialized, Role: FOLLOWER
I20260619 16:57:04.839274 15197 consensus_queue.cc:260] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:04.839471 15197 ts_tablet_manager.cc:1428] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:57:04.839529 15188 heartbeater.cc:503] Master 127.30.148.190:36793 was elected leader, sending a full tablet report...
W20260619 16:57:04.931865 15053 tablet.cc:2369] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:57:04.956365 14919 raft_consensus.cc:482] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:57:04.956513 14919 raft_consensus.cc:504] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:04.956857 14919 leader_election.cc:288] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007), 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427)
I20260619 16:57:04.959894 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.957148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:54191 (local address 127.30.148.131:45007)
0619 16:57:04.957206 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:04.957208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:04.957218 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:57:04.957251 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:04.957252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:04.957264 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:04.957340 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:04.957541 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.958083 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.959099 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.959487 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.959547 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.959554 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:04.959765 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 16:57:04.959805 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 16:57:04.959900 14915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.957003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:45007 (local address 127.30.148.129:54191)
0619 16:57:04.957064 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:04.957080 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:04.957158 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:04.957378 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:04.957379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:04.957382 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:04.957466 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.957468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.958158 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.958158 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.959005 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.959008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.959591 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.959593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.959672 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.959696 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 16:57:04.959751 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 16:57:04.959794 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 16:57:04.960031 15058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.957159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:46989 (local address 127.30.148.130:45427)
0619 16:57:04.957206 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:04.957208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:04.957384 (+ 176us) server_negotiation.cc:407] Connection header received
0619 16:57:04.957421 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:04.957422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:04.957449 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:04.957513 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:04.957690 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.958196 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.959154 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.959612 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.959689 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.959768 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:04.959890 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 16:57:04.959949 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 16:57:04.960042 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.957051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:45427 (local address 127.30.148.129:46989)
0619 16:57:04.957237 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:04.957246 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:04.957310 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:04.957543 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:04.957544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:04.957545 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:04.957617 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.957618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.958281 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.958282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.959023 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:04.959025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.959712 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.959713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:04.959799 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.959804 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:57:04.959870 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 16:57:04.959907 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":110,"threads_started":1}
I20260619 16:57:04.960291 15006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c9eee2c552478b8ae2c4ccddd12c6a" candidate_uuid: "18aa84a3feda4c939c4b5282191eeef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "517c6fe86f124376a4d18fd6e8544ff9" is_pre_election: true
I20260619 16:57:04.960449 15006 raft_consensus.cc:2454] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18aa84a3feda4c939c4b5282191eeef5 in term 0.
I20260619 16:57:04.960450 15142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c9eee2c552478b8ae2c4ccddd12c6a" candidate_uuid: "18aa84a3feda4c939c4b5282191eeef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" is_pre_election: true
I20260619 16:57:04.960645 15142 raft_consensus.cc:2454] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18aa84a3feda4c939c4b5282191eeef5 in term 0.
I20260619 16:57:04.960682 14799 leader_election.cc:302] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18aa84a3feda4c939c4b5282191eeef5, 517c6fe86f124376a4d18fd6e8544ff9; no voters:
I20260619 16:57:04.960807 14919 raft_consensus.cc:2781] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:57:04.960898 14919 raft_consensus.cc:482] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:57:04.960945 14919 raft_consensus.cc:3037] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:04.961717 14919 raft_consensus.cc:504] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:04.961859 14919 leader_election.cc:288] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 1 election: Requested vote from peers 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007), 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427)
I20260619 16:57:04.962039 15142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c9eee2c552478b8ae2c4ccddd12c6a" candidate_uuid: "18aa84a3feda4c939c4b5282191eeef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163"
I20260619 16:57:04.962044 15006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c9eee2c552478b8ae2c4ccddd12c6a" candidate_uuid: "18aa84a3feda4c939c4b5282191eeef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "517c6fe86f124376a4d18fd6e8544ff9"
I20260619 16:57:04.962128 15142 raft_consensus.cc:3037] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:04.962127 15006 raft_consensus.cc:3037] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:04.963037 15142 raft_consensus.cc:2454] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18aa84a3feda4c939c4b5282191eeef5 in term 1.
I20260619 16:57:04.963037 15006 raft_consensus.cc:2454] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18aa84a3feda4c939c4b5282191eeef5 in term 1.
I20260619 16:57:04.963246 14801 leader_election.cc:302] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18aa84a3feda4c939c4b5282191eeef5, 1fe2b12a7c8341c3a20d9c239c9ee163; no voters:
I20260619 16:57:04.963369 14919 raft_consensus.cc:2781] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:04.963557 14919 raft_consensus.cc:686] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 LEADER]: Becoming Leader. State: Replica: 18aa84a3feda4c939c4b5282191eeef5, State: Running, Role: LEADER
I20260619 16:57:04.963639 14919 consensus_queue.cc:237] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:04.964341 14724 catalog_manager.cc:5495] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18aa84a3feda4c939c4b5282191eeef5 (127.30.148.129). New cstate: current_term: 1 leader_uuid: "18aa84a3feda4c939c4b5282191eeef5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } health_report { overall_health: HEALTHY } } }
I20260619 16:57:04.976190 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.974105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.30.148.129:36209)
0619 16:57:04.974163 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:04.974164 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:04.974172 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:57:04.974200 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:04.974200 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:04.974216 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:04.974273 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:57:04.974367 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.974823 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.975513 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:04.975710 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:04.975767 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:04.975884 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:57:04.975980 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:57:04.976034 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:04.976074 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 16:57:04.976091 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 16:57:04.977705 15142 raft_consensus.cc:1261] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 1 FOLLOWER]: Refusing update from remote peer 18aa84a3feda4c939c4b5282191eeef5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:04.977725 15006 raft_consensus.cc:1261] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9 [term 1 FOLLOWER]: Refusing update from remote peer 18aa84a3feda4c939c4b5282191eeef5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:04.978024 15201 consensus_queue.cc:1035] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:04.978163 14919 consensus_queue.cc:1035] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:04.980093 15205 mvcc.cc:205] Tried to move back new op lower bound from 7298614169505796096 to 7298614169451569152. Current Snapshot: MvccSnapshot[applied={T|T < 7298614169505796096}]
I20260619 16:57:04.981148 15207 mvcc.cc:205] Tried to move back new op lower bound from 7298614169505796096 to 7298614169451569152. Current Snapshot: MvccSnapshot[applied={T|T < 7298614169505796096}]
I20260619 16:57:04.982023 14724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:04.982113 14724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:04.982834 14724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Sending DeleteTablet for 3 replicas of tablet 03c9eee2c552478b8ae2c4ccddd12c6a
I20260619 16:57:04.983165 15122 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c9eee2c552478b8ae2c4ccddd12c6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:04 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:37388
I20260619 16:57:04.983192 14986 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c9eee2c552478b8ae2c4ccddd12c6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:04 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:44200
I20260619 16:57:04.983276 15059 tablet_replica.cc:330] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9: stopping tablet replica
I20260619 16:57:04.983327 15059 raft_consensus.cc:2229] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:04.983321 14847 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c9eee2c552478b8ae2c4ccddd12c6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:04 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:44712
I20260619 16:57:04.983364 15059 raft_consensus.cc:2258] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:04.983371 14830 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c9eee2c552478b8ae2c4ccddd12c6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:04 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:44712
I20260619 16:57:04.983412 14922 tablet_replica.cc:330] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5: stopping tablet replica
I20260619 16:57:04.983467 14922 raft_consensus.cc:2229] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:57:04.983501 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 14763
I20260619 16:57:04.983551 14922 raft_consensus.cc:2258] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:04.983522 15214 tablet_replica.cc:330] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163: stopping tablet replica
I20260619 16:57:04.984220 15214 raft_consensus.cc:2229] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:04.984333 15214 raft_consensus.cc:2258] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:04.984401 15059 ts_tablet_manager.cc:1905] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:04.984768 15214 ts_tablet_manager.cc:1905] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:04.986030 15059 ts_tablet_manager.cc:1918] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:04.986090 15059 log.cc:1196] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/03c9eee2c552478b8ae2c4ccddd12c6a
I20260619 16:57:04.986303 15059 ts_tablet_manager.cc:1939] T 03c9eee2c552478b8ae2c4ccddd12c6a P 517c6fe86f124376a4d18fd6e8544ff9: Deleting consensus metadata
I20260619 16:57:04.986397 15214 ts_tablet_manager.cc:1918] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:04.986477 15214 log.cc:1196] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/03c9eee2c552478b8ae2c4ccddd12c6a
I20260619 16:57:04.986639 14708 catalog_manager.cc:4841] TS 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427): tablet 03c9eee2c552478b8ae2c4ccddd12c6a (table test-table [id=77d95b0411d842788ea9366ed23241d9]) successfully deleted
I20260619 16:57:04.986748 15214 ts_tablet_manager.cc:1939] T 03c9eee2c552478b8ae2c4ccddd12c6a P 1fe2b12a7c8341c3a20d9c239c9ee163: Deleting consensus metadata
I20260619 16:57:04.987109 14710 catalog_manager.cc:4841] TS 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007): tablet 03c9eee2c552478b8ae2c4ccddd12c6a (table test-table [id=77d95b0411d842788ea9366ed23241d9]) successfully deleted
W20260619 16:57:04.990100 14710 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:36209 (error 108)
I20260619 16:57:04.990159 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 14918
I20260619 16:57:04.991042 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:04.990904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36209 (local address 127.0.0.1:44730)
0619 16:57:04.990956 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:04.991005 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:36209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 16:57:04.991147 14710 catalog_manager.cc:4584] TS 18aa84a3feda4c939c4b5282191eeef5 (127.30.148.129:36209): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 03c9eee2c552478b8ae2c4ccddd12c6a: Network error: Client connection negotiation failed: client connection to 127.30.148.129:36209: connect: Connection refused (error 111)
I20260619 16:57:04.995540 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 15056
I20260619 16:57:05.000041 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 14691
I20260619 16:57:25.005242 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36793
--webserver_interface=127.30.148.190
--webserver_port=42921
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:36793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:25.083437 15216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:25.083688 15216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:25.083727 15216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:25.084852 15216 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 16:57:25.084919 15216 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 16:57:25.084940 15216 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 16:57:25.084959 15216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:25.084992 15216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:25.085016 15216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:25.085043 15216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:25.085064 15216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:25.086576 15216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:36793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=42921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:25.086884 15216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:25.087122 15216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:57:25.092686 15216 server_base.cc:1034] running on GCE node
W20260619 16:57:25.092636 15222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:25.092599 15224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:25.092599 15221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:25.093129 15216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:25.093461 15216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:25.094627 15216 hybrid_clock.cc:648] HybridClock initialized: now 1781888245094613 us; error 32 us; skew 500 ppm
Jun 19 16:57:25 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888245, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:25.101181 15216 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:57:25.101688 15216 webserver.cc:460] Webserver started at http://127.30.148.190:42921/ using document root <none> and password file <none>
I20260619 16:57:25.101926 15216 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:25.101996 15216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:25.103261 15216 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:57:25.103938 15231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:25.104146 15216 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:57:25.104249 15216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "d97b84b43b26421eb17dc43b5df1ecbc"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "a2f1ea3f9319263b6e6152d3e2ca4094"
server_key_iv: "fb009932bd70e217cc9e636919a3558b"
server_key_version: "encryptionkey@0"
I20260619 16:57:25.104588 15216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:25.119472 15216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:25.121240 15216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:25.121455 15216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:25.125047 15216 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:36793
I20260619 16:57:25.125082 15283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:36793 every 8 connection(s)
I20260619 16:57:25.125514 15216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:25.126904 15284 sys_catalog.cc:263] Verifying existing consensus state
I20260619 16:57:25.127542 15284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Bootstrap starting.
I20260619 16:57:25.130071 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 15216
I20260619 16:57:25.130427 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:36209
--local_ip_for_outbound_sockets=127.30.148.129
--tserver_master_addrs=127.30.148.190:36793
--webserver_port=45717
--webserver_interface=127.30.148.129
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:57:25.136194 15284 log.cc:824] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:25.138408 15284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:57:25.138715 15284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Bootstrap complete.
I20260619 16:57:25.140847 15284 raft_consensus.cc:348] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } }
I20260619 16:57:25.141057 15284 raft_consensus.cc:729] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97b84b43b26421eb17dc43b5df1ecbc, State: Initialized, Role: FOLLOWER
I20260619 16:57:25.141217 15284 consensus_queue.cc:260] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } }
I20260619 16:57:25.141275 15284 raft_consensus.cc:388] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:25.141337 15284 raft_consensus.cc:482] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:25.141397 15284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 2 FOLLOWER]: Advancing to term 3
I20260619 16:57:25.142343 15284 raft_consensus.cc:504] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } }
I20260619 16:57:25.142426 15284 leader_election.cc:302] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d97b84b43b26421eb17dc43b5df1ecbc; no voters:
I20260619 16:57:25.142653 15284 leader_election.cc:288] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 16:57:25.142688 15288 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 3 FOLLOWER]: Leader election won for term 3
I20260619 16:57:25.142798 15288 raft_consensus.cc:686] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [term 3 LEADER]: Becoming Leader. State: Replica: d97b84b43b26421eb17dc43b5df1ecbc, State: Running, Role: LEADER
I20260619 16:57:25.142910 15284 sys_catalog.cc:564] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:25.142938 15288 consensus_queue.cc:237] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } }
I20260619 16:57:25.143299 15290 sys_catalog.cc:455] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d97b84b43b26421eb17dc43b5df1ecbc. Latest consensus state: current_term: 3 leader_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } } }
I20260619 16:57:25.143311 15289 sys_catalog.cc:455] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97b84b43b26421eb17dc43b5df1ecbc" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36793 } } }
I20260619 16:57:25.143394 15290 sys_catalog.cc:458] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:25.143421 15289 sys_catalog.cc:458] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:25.143611 15295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:25.144570 15295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:25.144718 15295 catalog_manager.cc:1259] Loaded cluster ID: c9895b0ec9e94ddb8b834f839110174b
I20260619 16:57:25.144937 15295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:25.146118 15295 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:25.146320 15295 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Loaded TSK: 0
I20260619 16:57:25.146395 15295 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d97b84b43b26421eb17dc43b5df1ecbc: Loaded TSK: 1
F20260619 16:57:25.146663 15295 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc3d7ec59ad google::LogMessage::Fail() at ??:0
@ 0x7fc3d7ec888a google::LogMessage::SendToLog() at ??:0
@ 0x7fc3d7ec54de google::LogMessage::Flush() at ??:0
@ 0x7fc3d7ec6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc3da632104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc3da62fd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc3dfe231cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc3dfe2348e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc3dfe23594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fc3dfde9c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fc3dfdf3ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc3d9d930d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc3d9d88954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc3dabec6db start_thread at ??:0
@ 0x7fc3d69c361f clone at ??:0
W20260619 16:57:25.230331 15286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:25.230541 15286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:25.230566 15286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:25.231684 15286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:25.231724 15286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:25.231750 15286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:57:25.233134 15286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:36209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=45717
--tserver_master_addrs=127.30.148.190:36793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:25.233356 15286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:25.233628 15286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:57:25.238368 15286 server_base.cc:1034] running on GCE node
W20260619 16:57:25.238391 15326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:25.238507 15328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:25.238459 15325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:25.238767 15286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:25.239041 15286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:25.240195 15286 hybrid_clock.cc:648] HybridClock initialized: now 1781888245240168 us; error 43 us; skew 500 ppm
Jun 19 16:57:25 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888245, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:25.242605 15286 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:57:25.243079 15286 webserver.cc:460] Webserver started at http://127.30.148.129:45717/ using document root <none> and password file <none>
I20260619 16:57:25.243302 15286 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:25.243345 15286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:25.244613 15286 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:25.245285 15335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:25.245472 15286 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:25.245574 15286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "18aa84a3feda4c939c4b5282191eeef5"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "50bc5059e85298507082ba4714333fd1"
server_key_iv: "1278d3db870df7718b0e9665921080f5"
server_key_version: "encryptionkey@0"
I20260619 16:57:25.245966 15286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:25.274317 15286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:25.276950 15286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:25.277117 15286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:25.277390 15286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:25.277920 15342 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 16:57:25.279410 15286 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 16:57:25.279467 15286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 16:57:25.279500 15286 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 16:57:25.280385 15286 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 16:57:25.280427 15286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 16:57:25.280539 15342 tablet_bootstrap.cc:492] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5: Bootstrap starting.
I20260619 16:57:25.287528 15286 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:36209
I20260619 16:57:25.287704 15449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:36209 every 8 connection(s)
I20260619 16:57:25.287969 15286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:57:25.291304 15342 log.cc:824] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:25.293264 15342 tablet_bootstrap.cc:492] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:57:25.293584 15342 tablet_bootstrap.cc:492] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5: Bootstrap complete.
I20260619 16:57:25.294468 15342 ts_tablet_manager.cc:1397] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.010s
I20260619 16:57:25.295399 15342 raft_consensus.cc:348] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:25.295644 15342 raft_consensus.cc:729] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18aa84a3feda4c939c4b5282191eeef5, State: Initialized, Role: FOLLOWER
I20260619 16:57:25.295814 15342 consensus_queue.cc:260] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:25.295922 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 15286
I20260619 16:57:25.296101 15342 ts_tablet_manager.cc:1428] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 16:57:25.296231 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:45427
--local_ip_for_outbound_sockets=127.30.148.130
--tserver_master_addrs=127.30.148.190:36793
--webserver_port=35793
--webserver_interface=127.30.148.130
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:25.343894 15452 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:57:25.288636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:39161)
0619 16:57:25.288821 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:25.288864 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:25.289643 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:25.343698 (+ 54055us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
W20260619 16:57:25.344177 15450 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:36793 (0 consecutive failures): Network error: Failed to ping master at 127.30.148.190:36793: Client connection negotiation failed: client connection to 127.30.148.190:36793: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 16:57:25.344513 15452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:25.344402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:42835)
0619 16:57:25.344454 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:25.344496 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 16:57:25.344880 15452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:25.344783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:44111)
0619 16:57:25.344834 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:25.344860 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 16:57:25.345009 15450 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 16:57:25.377548 15455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:25.377753 15455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:25.377791 15455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:25.378919 15455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:25.378979 15455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:25.379024 15455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:57:25.380434 15455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:45427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=35793
--tserver_master_addrs=127.30.148.190:36793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:25.380689 15455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:25.380914 15455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:25.385825 15461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:25.385828 15460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:25.385855 15455 server_base.cc:1034] running on GCE node
W20260619 16:57:25.385835 15463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:25.386261 15455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:25.386493 15455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:25.387653 15455 hybrid_clock.cc:648] HybridClock initialized: now 1781888245387644 us; error 28 us; skew 500 ppm
Jun 19 16:57:25 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888245, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:25.390100 15455 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:57:25.390586 15455 webserver.cc:460] Webserver started at http://127.30.148.130:35793/ using document root <none> and password file <none>
I20260619 16:57:25.390811 15455 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:25.390875 15455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:25.392174 15455 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:57:25.392768 15470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:25.392953 15455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:57:25.393046 15455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "517c6fe86f124376a4d18fd6e8544ff9"
format_stamp: "Formatted at 2026-06-19 16:56:43 on dist-test-slave-xb9n"
server_key: "66b1278ddac990fa09feec09553792c3"
server_key_iv: "cf9c90705493fff1575faa13ed783574"
server_key_version: "encryptionkey@0"
I20260619 16:57:25.393396 15455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:25.422098 15455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:25.424379 15455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:25.424543 15455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:25.424765 15455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:25.425066 15455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:25.425118 15455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:25.425158 15455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:25.425184 15455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:25.430963 15455 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:45427
I20260619 16:57:25.431028 15583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:45427 every 8 connection(s)
I20260619 16:57:25.431324 15455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:57:25.432228 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:25.431896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:39835)
0619 16:57:25.432039 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:25.432102 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
W20260619 16:57:25.432590 15584 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:36793 (0 consecutive failures): Network error: Failed to ping master at 127.30.148.190:36793: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
I20260619 16:57:25.432816 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:25.432755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:51809)
0619 16:57:25.432780 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:25.432801 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260619 16:57:25.433039 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:25.432993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:47997)
0619 16:57:25.433013 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:25.433029 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260619 16:57:25.433136 15584 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 16:57:25.441555 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 15455
I20260619 16:57:25.441802 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:45007
--local_ip_for_outbound_sockets=127.30.148.131
--tserver_master_addrs=127.30.148.190:36793
--webserver_port=34813
--webserver_interface=127.30.148.131
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:25.519601 15587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:25.519789 15587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:25.519811 15587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:25.520862 15587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:25.520896 15587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:25.520922 15587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:57:25.522347 15587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:45007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=34813
--tserver_master_addrs=127.30.148.190:36793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:25.522519 15587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:25.522680 15587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:25.527606 15593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:25.527595 15592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:25.527586 15595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:25.527680 15587 server_base.cc:1034] running on GCE node
I20260619 16:57:25.528020 15587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:25.528267 15587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:25.529422 15587 hybrid_clock.cc:648] HybridClock initialized: now 1781888245529411 us; error 26 us; skew 500 ppm
Jun 19 16:57:25 dist-test-slave-xb9n krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888245, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:25.531901 15587 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:57:25.532313 15587 webserver.cc:460] Webserver started at http://127.30.148.131:34813/ using document root <none> and password file <none>
I20260619 16:57:25.532546 15587 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:25.532613 15587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:25.533970 15587 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:57:25.534560 15602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:25.534716 15587 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:57:25.534802 15587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "1fe2b12a7c8341c3a20d9c239c9ee163"
format_stamp: "Formatted at 2026-06-19 16:56:44 on dist-test-slave-xb9n"
server_key: "af66cc48e69ce4ce315bcf7146315c29"
server_key_iv: "1b6b5ae20f68d042440e26eea5dd37f8"
server_key_version: "encryptionkey@0"
I20260619 16:57:25.535125 15587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:25.568712 15587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:25.571002 15587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:25.571162 15587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:25.571385 15587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:25.571682 15587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:25.571733 15587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:25.571774 15587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:25.571801 15587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:25.577571 15587 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:45007
I20260619 16:57:25.577618 15715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:45007 every 8 connection(s)
I20260619 16:57:25.577916 15587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:57:25.578827 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:25.578489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:53659)
0619 16:57:25.578634 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:25.578698 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
W20260619 16:57:25.579245 15716 heartbeater.cc:644] Failed to heartbeat to 127.30.148.190:36793 (0 consecutive failures): Network error: Failed to ping master at 127.30.148.190:36793: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
I20260619 16:57:25.579505 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:25.579414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:34773)
0619 16:57:25.579460 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:25.579484 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 16:57:25.579775 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:25.579694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:36027)
0619 16:57:25.579735 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:25.579756 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 16:57:25.579879 15716 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 16:57:25.586495 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 15587
I20260619 16:57:26.345518 15720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:26.345297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:33119)
0619 16:57:26.345430 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:26.345463 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":100,"threads_started":1}
I20260619 16:57:26.433652 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:26.433454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:53301)
0619 16:57:26.433556 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:26.433587 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
I20260619 16:57:26.580354 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:26.580153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:60355)
0619 16:57:26.580276 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:26.580309 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":53,"threads_started":1}
I20260619 16:57:26.679200 15723 raft_consensus.cc:482] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:57:26.679314 15723 raft_consensus.cc:504] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:26.679723 15723 leader_election.cc:288] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007), 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427)
Jun 19 16:57:26 dist-test-slave-xb9n krb5kdc[14146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888245, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.130@KRBTEST.COM
Jun 19 16:57:26 dist-test-slave-xb9n krb5kdc[14146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888245, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.131@KRBTEST.COM
I20260619 16:57:26.690025 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:26.680015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:33913 (local address 127.30.148.130:45427)
0619 16:57:26.680143 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:26.680157 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:26.680194 (+ 37us) server_negotiation.cc:407] Connection header received
0619 16:57:26.680711 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:26.680714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:26.680756 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:26.680862 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:26.681656 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:26.682469 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:26.683503 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:26.683878 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:26.684001 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:26.686359 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:26.686363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:26.686363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:26.686374 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:26.688222 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:26.688732 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:26.688735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:26.688736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:26.688838 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:26.689112 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:26.689113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:26.689114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:26.689337 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:26.689477 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:26.689771 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 16:57:26.689885 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":45,"threads_started":1}
I20260619 16:57:26.690017 15720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:26.679842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:45007 (local address 127.30.148.129:54241)
0619 16:57:26.679910 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:26.679921 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:26.679973 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:26.680933 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:26.680935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:26.681319 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:26.681534 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:26.681540 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:26.682701 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:26.682701 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:26.683358 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:26.683360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:26.683951 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:26.683952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:26.684074 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:26.684626 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:26.684633 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:26.686581 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:26.688480 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:26.688481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:26.688481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:26.688708 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:26.688971 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:26.688972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:26.688972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:26.689047 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:26.689496 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:26.689497 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:26.689607 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 16:57:26.689718 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 16:57:26.689896 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":35}
I20260619 16:57:26.690135 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:26.679989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:54241 (local address 127.30.148.131:45007)
0619 16:57:26.680183 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:26.680195 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:26.680227 (+ 32us) server_negotiation.cc:407] Connection header received
0619 16:57:26.680718 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:26.680720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:26.680758 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:26.680904 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:26.681676 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:26.682608 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:26.683475 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:26.683866 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:26.683966 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:26.686682 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:26.686685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:26.686685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:26.686696 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:26.688396 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:26.688788 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:26.688790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:26.688790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:26.688898 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:26.689132 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:26.689133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:26.689133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:26.689396 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:26.689503 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:26.689773 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 16:57:26.690015 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":102,"threads_started":1}
I20260619 16:57:26.690177 15724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:26.679892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:45427 (local address 127.30.148.129:33913)
0619 16:57:26.680110 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:26.680123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:26.680180 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:26.680961 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:26.680962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:26.681304 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:26.681533 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:26.681539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:26.682557 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:26.682558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:26.683359 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:26.683361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:26.683980 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:26.683981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:26.684072 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:26.684677 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:26.684682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:26.686224 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:26.688316 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:26.688323 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:26.688325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:26.688653 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:26.688906 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:26.688907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:26.688907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:26.689035 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:26.689473 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:26.689480 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:26.689607 (+ 127us) client_negotiation.cc:769] Sending connection context
0619 16:57:26.689717 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 16:57:26.690008 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":126,"thread_start_us":64,"threads_started":1}
W20260619 16:57:26.690904 15338 leader_election.cc:341] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007): Not found: Tablet not found: 03c9eee2c552478b8ae2c4ccddd12c6a
W20260619 16:57:26.691359 15336 leader_election.cc:341] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427): Not found: Tablet not found: 03c9eee2c552478b8ae2c4ccddd12c6a
I20260619 16:57:26.691411 15336 leader_election.cc:302] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18aa84a3feda4c939c4b5282191eeef5; no voters: 1fe2b12a7c8341c3a20d9c239c9ee163, 517c6fe86f124376a4d18fd6e8544ff9
I20260619 16:57:26.691517 15723 raft_consensus.cc:2726] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 16:57:27.346100 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:27.345905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:38125)
0619 16:57:27.346025 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:27.346054 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260619 16:57:27.434226 15728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:27.434044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:34535)
0619 16:57:27.434151 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:27.434185 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260619 16:57:27.581000 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:27.580787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:45285)
0619 16:57:27.580930 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:27.580965 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":57,"threads_started":1}
I20260619 16:57:28.346709 15730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:28.346533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:43859)
0619 16:57:28.346634 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:28.346665 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":53,"threads_started":1}
I20260619 16:57:28.434871 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:28.434673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:43079)
0619 16:57:28.434801 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:28.434836 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260619 16:57:28.455511 15732 raft_consensus.cc:482] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:57:28.455593 15732 raft_consensus.cc:504] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:28.455741 15732 leader_election.cc:288] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007), 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427)
W20260619 16:57:28.456140 15338 leader_election.cc:341] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007): Not found: Tablet not found: 03c9eee2c552478b8ae2c4ccddd12c6a
W20260619 16:57:28.456226 15336 leader_election.cc:341] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427): Not found: Tablet not found: 03c9eee2c552478b8ae2c4ccddd12c6a
I20260619 16:57:28.456311 15338 leader_election.cc:302] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18aa84a3feda4c939c4b5282191eeef5; no voters: 1fe2b12a7c8341c3a20d9c239c9ee163, 517c6fe86f124376a4d18fd6e8544ff9
I20260619 16:57:28.456393 15732 raft_consensus.cc:2726] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 16:57:28.581686 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:28.581502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:52643)
0619 16:57:28.581619 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:28.581651 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260619 16:57:29.347411 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:29.347167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:44145)
0619 16:57:29.347310 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:29.347356 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
I20260619 16:57:29.435577 15735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:29.435372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:50511)
0619 16:57:29.435493 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:29.435532 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260619 16:57:29.582396 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:29.582204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:44245)
0619 16:57:29.582319 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:29.582360 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":54,"threads_started":1}
I20260619 16:57:30.233609 15737 raft_consensus.cc:482] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:57:30.233706 15737 raft_consensus.cc:504] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:30.233858 15737 leader_election.cc:288] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007), 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427)
W20260619 16:57:30.234216 15338 leader_election.cc:341] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007): Not found: Tablet not found: 03c9eee2c552478b8ae2c4ccddd12c6a
W20260619 16:57:30.234288 15336 leader_election.cc:341] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427): Not found: Tablet not found: 03c9eee2c552478b8ae2c4ccddd12c6a
I20260619 16:57:30.234381 15336 leader_election.cc:302] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18aa84a3feda4c939c4b5282191eeef5; no voters: 1fe2b12a7c8341c3a20d9c239c9ee163, 517c6fe86f124376a4d18fd6e8544ff9
I20260619 16:57:30.234573 15737 raft_consensus.cc:2726] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 16:57:30.348112 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:30.347874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:47059)
0619 16:57:30.348014 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:30.348068 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":57,"threads_started":1}
I20260619 16:57:30.436167 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:30.435944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:49395)
0619 16:57:30.436068 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:30.436101 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260619 16:57:30.583169 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:30.583009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:56123)
0619 16:57:30.583110 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:30.583134 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":50,"threads_started":1}
I20260619 16:57:31.348764 15741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:31.348560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:42985)
0619 16:57:31.348685 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:31.348719 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260619 16:57:31.436761 15742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:31.436576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:50429)
0619 16:57:31.436692 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:31.436725 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260619 16:57:31.583788 15743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:31.583614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:52967)
0619 16:57:31.583723 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:31.583751 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260619 16:57:32.349629 15744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:32.349427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:50725)
0619 16:57:32.349548 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:32.349584 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260619 16:57:32.437582 15745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:32.437342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:44949)
0619 16:57:32.437515 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:32.437547 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":74,"threads_started":1}
I20260619 16:57:32.578269 15746 raft_consensus.cc:482] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:57:32.578377 15746 raft_consensus.cc:504] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:32.578588 15746 leader_election.cc:288] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007), 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427)
W20260619 16:57:32.578882 15338 leader_election.cc:341] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007): Not found: Tablet not found: 03c9eee2c552478b8ae2c4ccddd12c6a
W20260619 16:57:32.578976 15336 leader_election.cc:341] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427): Not found: Tablet not found: 03c9eee2c552478b8ae2c4ccddd12c6a
I20260619 16:57:32.579054 15336 leader_election.cc:302] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18aa84a3feda4c939c4b5282191eeef5; no voters: 1fe2b12a7c8341c3a20d9c239c9ee163, 517c6fe86f124376a4d18fd6e8544ff9
I20260619 16:57:32.579133 15746 raft_consensus.cc:2726] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 16:57:32.584367 15747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:32.584189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:36245)
0619 16:57:32.584303 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:32.584334 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260619 16:57:33.350288 15748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:33.350087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:53365)
0619 16:57:33.350201 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:33.350232 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260619 16:57:33.438179 15749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:33.438005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:56815)
0619 16:57:33.438113 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:33.438144 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260619 16:57:33.584928 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:33.584737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:44553)
0619 16:57:33.584861 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:33.584893 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260619 16:57:34.350865 15751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:34.350653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:42475)
0619 16:57:34.350785 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:34.350821 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260619 16:57:34.438784 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:34.438574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:60191)
0619 16:57:34.438715 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:34.438750 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260619 16:57:34.585587 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:34.585404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:53595)
0619 16:57:34.585519 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:34.585550 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260619 16:57:35.351485 15754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:35.351281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:35609)
0619 16:57:35.351409 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:35.351442 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260619 16:57:35.439383 15755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:35.439209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:53813)
0619 16:57:35.439315 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:35.439348 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":53,"threads_started":1}
I20260619 16:57:35.586201 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:35.586010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:49863)
0619 16:57:35.586129 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:35.586162 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
I20260619 16:57:36.352087 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:36.351869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:32941)
0619 16:57:36.351999 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:36.352034 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":57,"threads_started":1}
I20260619 16:57:36.439966 15758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:36.439789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:47979)
0619 16:57:36.439897 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:36.439929 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":54,"threads_started":1}
I20260619 16:57:36.586867 15759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:36.586656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:34745)
0619 16:57:36.586796 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:36.586829 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260619 16:57:37.352747 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:37.352516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:48321)
0619 16:57:37.352666 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:37.352700 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":64,"threads_started":1}
I20260619 16:57:37.440542 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:37.440371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:50973)
0619 16:57:37.440475 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:37.440508 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260619 16:57:37.587576 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:37.587376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:36519)
0619 16:57:37.587504 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:37.587539 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260619 16:57:37.699010 15763 raft_consensus.cc:482] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:57:37.699142 15763 raft_consensus.cc:504] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe2b12a7c8341c3a20d9c239c9ee163" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45007 } } peers { permanent_uuid: "517c6fe86f124376a4d18fd6e8544ff9" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45427 } } peers { permanent_uuid: "18aa84a3feda4c939c4b5282191eeef5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36209 } }
I20260619 16:57:37.699311 15763 leader_election.cc:288] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007), 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427)
W20260619 16:57:37.699697 15336 leader_election.cc:341] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 517c6fe86f124376a4d18fd6e8544ff9 (127.30.148.130:45427): Not found: Tablet not found: 03c9eee2c552478b8ae2c4ccddd12c6a
W20260619 16:57:37.699806 15338 leader_election.cc:341] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1fe2b12a7c8341c3a20d9c239c9ee163 (127.30.148.131:45007): Not found: Tablet not found: 03c9eee2c552478b8ae2c4ccddd12c6a
I20260619 16:57:37.699992 15338 leader_election.cc:302] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18aa84a3feda4c939c4b5282191eeef5; no voters: 1fe2b12a7c8341c3a20d9c239c9ee163, 517c6fe86f124376a4d18fd6e8544ff9
I20260619 16:57:37.700064 15763 raft_consensus.cc:2726] T 03c9eee2c552478b8ae2c4ccddd12c6a P 18aa84a3feda4c939c4b5282191eeef5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 16:57:38.353396 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:38.353164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:58807)
0619 16:57:38.353270 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:38.353300 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
I20260619 16:57:38.441294 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:38.441079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:50849)
0619 16:57:38.441212 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:38.441246 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":56,"threads_started":1}
I20260619 16:57:38.588624 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:38.587964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:39081)
0619 16:57:38.588549 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:38.588578 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":515,"threads_started":1}
I20260619 16:57:39.354024 15767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:39.353838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:47537)
0619 16:57:39.353945 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:39.353979 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260619 16:57:39.442006 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:39.441799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:49441)
0619 16:57:39.441930 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:39.441966 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
I20260619 16:57:39.589335 15769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:39.589116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:44577)
0619 16:57:39.589243 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:39.589294 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":55,"threads_started":1}
I20260619 16:57:40.354852 15770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:40.354515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.129:37473)
0619 16:57:40.354765 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:40.354810 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":79,"threads_started":1}
I20260619 16:57:40.442760 15771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:40.442550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.130:60133)
0619 16:57:40.442686 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:40.442720 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260619 16:57:40.590029 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:40.589845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36793 (local address 127.30.148.131:45373)
0619 16:57:40.589961 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:40.589990 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.190:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260619 16:57:40.594734 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 15286
I20260619 16:57:40.601159 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 15455
I20260619 16:57:40.606468 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 15587
2026-06-19T16:57:40Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58329 ms)
[----------] 30 tests from SecurityITest (223683 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:57:40 dist-test-slave-xb9n krb5kdc[15775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:57:40 dist-test-slave-xb9n krb5kdc[15775](info): set up 2 sockets
Jun 19 16:57:40 dist-test-slave-xb9n krb5kdc[15775](info): commencing operation
krb5kdc: starting...
W20260619 16:57:41.648949 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:57:41 dist-test-slave-xb9n krb5kdc[15775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:57:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:57:41.756363 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36263
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:41717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:36263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:41.833026 15788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:41.833211 15788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:41.833232 15788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:41.834259 15788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:41.834293 15788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:41.834306 15788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:41.834318 15788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:41.834331 15788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:41.835659 15788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:41717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:36263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:41.835819 15788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:41.835973 15788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:41.840431 15793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:41.840444 15794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:41.840432 15796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:41.840699 15788 server_base.cc:1034] running on GCE node
I20260619 16:57:41.840930 15788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:41.841203 15788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:41.842340 15788 hybrid_clock.cc:648] HybridClock initialized: now 1781888261842317 us; error 42 us; skew 500 ppm
Jun 19 16:57:41 dist-test-slave-xb9n krb5kdc[15775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888261, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:41.844571 15788 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:57:41.844911 15788 webserver.cc:460] Webserver started at http://127.30.148.190:44849/ using document root <none> and password file <none>
I20260619 16:57:41.845104 15788 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:41.845141 15788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:41.845220 15788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:41.846210 15788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "92afb10cb21145e19692f7f9b435ed7a"
format_stamp: "Formatted at 2026-06-19 16:57:41 on dist-test-slave-xb9n"
server_key: "72eb1775e0f900a5ef1fcdb24c334cd5"
server_key_iv: "1eec97f3300cb3ae6062575b3f177e9b"
server_key_version: "encryptionkey@0"
I20260619 16:57:41.846508 15788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "92afb10cb21145e19692f7f9b435ed7a"
format_stamp: "Formatted at 2026-06-19 16:57:41 on dist-test-slave-xb9n"
server_key: "72eb1775e0f900a5ef1fcdb24c334cd5"
server_key_iv: "1eec97f3300cb3ae6062575b3f177e9b"
server_key_version: "encryptionkey@0"
I20260619 16:57:41.847654 15788 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:57:41.848332 15803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:41.848486 15788 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:41.848569 15788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "92afb10cb21145e19692f7f9b435ed7a"
format_stamp: "Formatted at 2026-06-19 16:57:41 on dist-test-slave-xb9n"
server_key: "72eb1775e0f900a5ef1fcdb24c334cd5"
server_key_iv: "1eec97f3300cb3ae6062575b3f177e9b"
server_key_version: "encryptionkey@0"
I20260619 16:57:41.848644 15788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:41.859927 15788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:41.861765 15788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:41.861904 15788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:41.865517 15788 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:36263
I20260619 16:57:41.865576 15855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:36263 every 8 connection(s)
I20260619 16:57:41.865911 15788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:41.866685 15856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:41.868763 15856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a: Bootstrap starting.
I20260619 16:57:41.869298 15856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:41.869602 15856 log.cc:824] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:41.870262 15856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a: No bootstrap required, opened a new log
I20260619 16:57:41.870376 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 15788
I20260619 16:57:41.870471 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:41.870669 31314 external_mini_cluster.cc:1428] Setting key 58c13d5fcad32a8fc535e798661966ff
I20260619 16:57:41.872187 15856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92afb10cb21145e19692f7f9b435ed7a" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36263 } }
I20260619 16:57:41.872354 15856 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:41.872394 15856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92afb10cb21145e19692f7f9b435ed7a, State: Initialized, Role: FOLLOWER
I20260619 16:57:41.872584 15856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92afb10cb21145e19692f7f9b435ed7a" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36263 } }
I20260619 16:57:41.872678 15856 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:41.872712 15856 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:41.872767 15856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:41.873718 15856 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92afb10cb21145e19692f7f9b435ed7a" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36263 } }
I20260619 16:57:41.873835 15856 leader_election.cc:302] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92afb10cb21145e19692f7f9b435ed7a; no voters:
I20260619 16:57:41.874028 15856 leader_election.cc:288] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:41.874060 15861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:41.874158 15861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [term 1 LEADER]: Becoming Leader. State: Replica: 92afb10cb21145e19692f7f9b435ed7a, State: Running, Role: LEADER
I20260619 16:57:41.874275 15856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:41.874295 15861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92afb10cb21145e19692f7f9b435ed7a" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36263 } }
I20260619 16:57:41.874593 15861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92afb10cb21145e19692f7f9b435ed7a. Latest consensus state: current_term: 1 leader_uuid: "92afb10cb21145e19692f7f9b435ed7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92afb10cb21145e19692f7f9b435ed7a" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36263 } } }
I20260619 16:57:41.874600 15862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92afb10cb21145e19692f7f9b435ed7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92afb10cb21145e19692f7f9b435ed7a" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36263 } } }
I20260619 16:57:41.874740 15861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:41.874962 15862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:41.875121 15867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:41.875598 15867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:41.877209 15867 catalog_manager.cc:1347] Generated new cluster ID: 306f0cd6e956498da588b6594775e92f
I20260619 16:57:41.877259 15867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:57:41 dist-test-slave-xb9n krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:41.882536 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:41.871143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44402 (local address 127.30.148.190:36263)
0619 16:57:41.871312 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:41.871319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:41.871341 (+ 22us) server_negotiation.cc:407] Connection header received
0619 16:57:41.872173 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:41.872179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:41.872223 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:41.872328 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:41.873138 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:41.874210 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:41.875309 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:41.875939 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:41.876046 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:41.878982 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:41.878985 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:41.878986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:41.878996 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:41.880879 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:41.881274 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:41.881277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:41.881277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:41.881407 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:41.881621 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:41.881622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:41.881623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:41.881899 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:41.882037 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:41.882141 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 16:57:41.882257 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260619 16:57:41.902432 15867 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:41.903086 15867 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:41.908537 15867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a: Generated new TSK 0
I20260619 16:57:41.908701 15867 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:57:41.941597 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36263
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:41717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:42.021466 15882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:42.021693 15882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:42.021733 15882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:42.022883 15882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:42.022944 15882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:42.022987 15882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:57:42.024430 15882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:41717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:42.024684 15882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:42.024894 15882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:42.029795 15890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:42.029807 15887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:42.029807 15888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:42.029841 15882 server_base.cc:1034] running on GCE node
I20260619 16:57:42.030102 15882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:42.030342 15882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:42.031493 15882 hybrid_clock.cc:648] HybridClock initialized: now 1781888262031476 us; error 30 us; skew 500 ppm
Jun 19 16:57:42 dist-test-slave-xb9n krb5kdc[15775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888262, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:42.033900 15882 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:57:42.034301 15882 webserver.cc:460] Webserver started at http://127.30.148.129:39971/ using document root <none> and password file <none>
I20260619 16:57:42.034503 15882 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:42.034564 15882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:42.034701 15882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:42.035708 15882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "2f83a1982c264585a4428a90dfd160bd"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "a618aec8d0073f6737dec7d5a383e67d"
server_key_iv: "3702cada08e9d1dfd2d7c5d0a0dd1d89"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.036043 15882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "2f83a1982c264585a4428a90dfd160bd"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "a618aec8d0073f6737dec7d5a383e67d"
server_key_iv: "3702cada08e9d1dfd2d7c5d0a0dd1d89"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.037330 15882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:57:42.038012 15897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.038178 15882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:42.038273 15882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "2f83a1982c264585a4428a90dfd160bd"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "a618aec8d0073f6737dec7d5a383e67d"
server_key_iv: "3702cada08e9d1dfd2d7c5d0a0dd1d89"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.038342 15882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:42.047992 15882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:42.050158 15882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:42.050303 15882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:42.050503 15882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:42.050787 15882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:42.050834 15882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.050871 15882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:42.050899 15882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.056527 15882 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:35691
I20260619 16:57:42.056599 16010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:35691 every 8 connection(s)
I20260619 16:57:42.056865 15882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
Jun 19 16:57:42 dist-test-slave-xb9n krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888262, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:42.065300 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.057535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:38199 (local address 127.30.148.190:36263)
0619 16:57:42.057575 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:42.057578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:42.057789 (+ 211us) server_negotiation.cc:407] Connection header received
0619 16:57:42.058251 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:42.058251 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:42.058263 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:42.058313 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:42.059115 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.059581 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.060480 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.060669 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.060717 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.062746 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:42.062748 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:42.062748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:42.062756 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:42.064163 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:42.064497 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:42.064498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:42.064499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:42.064555 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:42.064765 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:42.064765 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:42.064766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:42.064864 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:42.064915 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:42.065167 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 16:57:42.065238 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:57:42.065445 16013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.057508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36263 (local address 127.30.148.129:38199)
0619 16:57:42.057659 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:42.057699 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:42.058175 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:42.058359 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:42.058362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:42.058870 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:42.059038 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.059043 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.059648 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.059648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.060411 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.060413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.060734 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.060735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.060829 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.061236 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:42.061244 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:42.062649 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:42.064229 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:42.064234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:42.064236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:42.064420 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:42.064619 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:42.064619 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:42.064620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:42.064712 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:42.064921 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:42.064924 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:42.065039 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 16:57:42.065144 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 16:57:42.065249 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260619 16:57:42.065888 16011 heartbeater.cc:347] Connected to a master server at 127.30.148.190:36263
I20260619 16:57:42.065989 16011 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:42.066002 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 15882
I20260619 16:57:42.066099 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:57:42.066185 16011 heartbeater.cc:511] Master 127.30.148.190:36263 requested a full tablet report, sending...
I20260619 16:57:42.066325 31314 external_mini_cluster.cc:1428] Setting key 8c3284e2fa2d154d1df4edff89a9cc57
I20260619 16:57:42.066738 15820 ts_manager.cc:194] Registered new tserver with Master: 2f83a1982c264585a4428a90dfd160bd (127.30.148.129:35691)
I20260619 16:57:42.067402 15820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:38199
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:57:42.089448 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36263
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:41717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:42.168131 16016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:42.168315 16016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:42.168337 16016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:42.169490 16016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:42.169528 16016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:42.169554 16016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:57:42.170928 16016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:41717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:42.171141 16016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:42.171347 16016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:42.176110 16021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:42.176081 16024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:42.176151 16022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:42.176355 16016 server_base.cc:1034] running on GCE node
I20260619 16:57:42.176539 16016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:42.176735 16016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:42.177896 16016 hybrid_clock.cc:648] HybridClock initialized: now 1781888262177882 us; error 35 us; skew 500 ppm
Jun 19 16:57:42 dist-test-slave-xb9n krb5kdc[15775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888262, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:42.180280 16016 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:57:42.180677 16016 webserver.cc:460] Webserver started at http://127.30.148.130:38471/ using document root <none> and password file <none>
I20260619 16:57:42.180886 16016 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:42.180948 16016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:42.181088 16016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:42.182106 16016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "bc0d0cb8a27c49a39335a247115cabad"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "21c58f87f21c4c5e285dc58695d54ea1"
server_key_iv: "c44f568a611461c96f07abd6f9b15c91"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.182440 16016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "bc0d0cb8a27c49a39335a247115cabad"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "21c58f87f21c4c5e285dc58695d54ea1"
server_key_iv: "c44f568a611461c96f07abd6f9b15c91"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.183725 16016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:57:42.184389 16031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.184533 16016 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:57:42.184634 16016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "bc0d0cb8a27c49a39335a247115cabad"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "21c58f87f21c4c5e285dc58695d54ea1"
server_key_iv: "c44f568a611461c96f07abd6f9b15c91"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.184702 16016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:42.201674 16016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:42.203929 16016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:42.204069 16016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:42.204267 16016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:42.204547 16016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:42.204604 16016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.204643 16016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:42.204671 16016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.210409 16016 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:40887
I20260619 16:57:42.210453 16144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:40887 every 8 connection(s)
I20260619 16:57:42.210772 16016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:57:42.213853 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 16016
I20260619 16:57:42.213958 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:57:42.214169 31314 external_mini_cluster.cc:1428] Setting key 0befa5add83666740277efacbfff648b
Jun 19 16:57:42 dist-test-slave-xb9n krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888262, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:42.220629 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.211440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:42225 (local address 127.30.148.190:36263)
0619 16:57:42.211484 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:42.211486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:42.211661 (+ 175us) server_negotiation.cc:407] Connection header received
0619 16:57:42.212111 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:42.212112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:42.212122 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:42.212171 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:42.213003 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.213524 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.214904 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.215101 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.215157 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.217800 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:42.217802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:42.217802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:42.217807 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:42.219247 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:42.219725 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:42.219726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:42.219726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:42.219774 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:42.220046 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:42.220046 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:42.220047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:42.220146 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:42.220194 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:42.220508 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 16:57:42.220570 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:57:42.220894 16147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.211416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36263 (local address 127.30.148.130:42225)
0619 16:57:42.211565 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:42.211598 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:42.212019 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:42.212222 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:42.212225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:42.212750 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:42.212925 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.212929 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.213600 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.213600 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.214773 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.214775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.215178 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.215179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.215333 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.215789 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:42.215800 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:42.217702 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:42.219329 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:42.219336 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:42.219338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:42.219650 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:42.219837 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:42.219838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:42.219838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:42.219981 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:42.220206 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:42.220212 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:42.220364 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 16:57:42.220483 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 16:57:42.220635 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260619 16:57:42.221405 16145 heartbeater.cc:347] Connected to a master server at 127.30.148.190:36263
I20260619 16:57:42.221490 16145 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:42.221673 16145 heartbeater.cc:511] Master 127.30.148.190:36263 requested a full tablet report, sending...
I20260619 16:57:42.222040 15820 ts_manager.cc:194] Registered new tserver with Master: bc0d0cb8a27c49a39335a247115cabad (127.30.148.130:40887)
I20260619 16:57:42.222398 15820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:42225
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:57:42.239683 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36263
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:41717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:42.317507 16150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:42.317744 16150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:42.317780 16150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:42.318897 16150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:42.318959 16150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:42.319003 16150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:57:42.320430 16150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:41717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:36263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:42.320674 16150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:42.320889 16150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:42.325589 16155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:42.325654 16156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:42.325590 16158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:42.325915 16150 server_base.cc:1034] running on GCE node
I20260619 16:57:42.326074 16150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:42.326259 16150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:42.327425 16150 hybrid_clock.cc:648] HybridClock initialized: now 1781888262327416 us; error 32 us; skew 500 ppm
Jun 19 16:57:42 dist-test-slave-xb9n krb5kdc[15775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888262, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:42.329782 16150 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:57:42.330183 16150 webserver.cc:460] Webserver started at http://127.30.148.131:33973/ using document root <none> and password file <none>
I20260619 16:57:42.330365 16150 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:42.330403 16150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:42.330487 16150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:42.331390 16150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "a4000bb9ef3b4930bf4de1df7995edc5"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "2b03dc9e4b3ba436a91a586372a4a911"
server_key_iv: "2e419c5e852d2e80f9e5acedbe666e2d"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.331668 16150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "a4000bb9ef3b4930bf4de1df7995edc5"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "2b03dc9e4b3ba436a91a586372a4a911"
server_key_iv: "2e419c5e852d2e80f9e5acedbe666e2d"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.332818 16150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:57:42.333464 16165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.333607 16150 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.333690 16150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "a4000bb9ef3b4930bf4de1df7995edc5"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "2b03dc9e4b3ba436a91a586372a4a911"
server_key_iv: "2e419c5e852d2e80f9e5acedbe666e2d"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.333741 16150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:42.350901 16150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:42.352850 16150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:42.352953 16150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:42.353112 16150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:42.353480 16150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:42.353515 16150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.353549 16150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:42.353572 16150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.358960 16150 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:42427
I20260619 16:57:42.358999 16278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:42427 every 8 connection(s)
I20260619 16:57:42.359313 16150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:57:42.363840 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 16150
I20260619 16:57:42.363943 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:57:42.364168 31314 external_mini_cluster.cc:1428] Setting key 0129f6b461118e1c83307249588e833b
Jun 19 16:57:42 dist-test-slave-xb9n krb5kdc[15775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888262, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:42.368192 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.359993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:46691 (local address 127.30.148.190:36263)
0619 16:57:42.360035 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:42.360037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:42.360204 (+ 167us) server_negotiation.cc:407] Connection header received
0619 16:57:42.360680 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:42.360681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:42.360694 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:42.360746 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:42.361670 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.362169 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.363205 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.363418 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.363496 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.365777 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:42.365778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:42.365779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:42.365786 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:42.366900 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:42.367274 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:42.367275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:42.367275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:42.367332 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:42.367571 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:42.367571 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:42.367572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:42.367676 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:42.367742 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:42.367990 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 16:57:42.368073 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:57:42.368387 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.359954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36263 (local address 127.30.148.131:46691)
0619 16:57:42.360090 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:42.360131 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:42.360584 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:42.360809 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:42.360813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:42.361345 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:42.361586 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.361591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.362262 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.362263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.363104 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.363106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.363504 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.363505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.363625 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.364200 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:42.364210 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:42.365689 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:42.366988 (+ 1299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:42.366994 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:42.366996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:42.367202 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:42.367394 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:42.367395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:42.367396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:42.367509 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:42.367744 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:42.367750 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:42.367873 (+ 123us) client_negotiation.cc:769] Sending connection context
0619 16:57:42.367968 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 16:57:42.368135 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
I20260619 16:57:42.368814 16279 heartbeater.cc:347] Connected to a master server at 127.30.148.190:36263
I20260619 16:57:42.368912 16279 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:42.369074 16279 heartbeater.cc:511] Master 127.30.148.190:36263 requested a full tablet report, sending...
I20260619 16:57:42.369443 15820 ts_manager.cc:194] Registered new tserver with Master: a4000bb9ef3b4930bf4de1df7995edc5 (127.30.148.131:42427)
I20260619 16:57:42.369789 15820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:46691
I20260619 16:57:42.375947 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:57:42.382733 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.377244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44416 (local address 127.30.148.190:36263)
0619 16:57:42.377283 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:42.377285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:42.377376 (+ 91us) server_negotiation.cc:407] Connection header received
0619 16:57:42.377447 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:42.377448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:42.377461 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:42.377510 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:42.378038 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.378477 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.379167 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.379358 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.379411 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.380489 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:42.380490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:42.380491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:42.380496 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:42.381834 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:42.382134 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:42.382134 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:42.382135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:42.382178 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:42.382332 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:42.382333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:42.382333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:42.382459 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:42.382529 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:42.382604 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 16:57:42.382656 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:57:42.384100 15820 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:57:42.385556 15820 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:57:42.391499 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.387415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:42427 (local address 127.0.0.1:51750)
0619 16:57:42.387633 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:42.387672 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:42.387746 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:42.387954 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:42.387956 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:42.387958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:42.388021 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.388023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.389329 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.389330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.390390 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.390392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.391194 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.391195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.391280 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.391293 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 16:57:42.391373 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 16:57:42.391424 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":73,"thread_start_us":43,"threads_started":1}
I20260619 16:57:42.391520 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.387488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51750 (local address 127.30.148.131:42427)
0619 16:57:42.387637 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:42.387642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:42.387737 (+ 95us) server_negotiation.cc:407] Connection header received
0619 16:57:42.387835 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:42.387838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:42.387852 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:42.387925 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:42.388148 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.389206 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.390474 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.391112 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.391187 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.391235 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:42.391396 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 16:57:42.391443 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260619 16:57:42.391717 16287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.387457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:35691 (local address 127.0.0.1:33698)
0619 16:57:42.387864 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:42.387874 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:42.387928 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:42.388352 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:42.388353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:42.388355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:42.388389 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.388391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.389624 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.389624 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.390716 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.390718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.391474 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.391475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.391534 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.391549 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 16:57:42.391603 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 16:57:42.391648 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":37,"threads_started":1}
I20260619 16:57:42.391852 16288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.387446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:40887 (local address 127.0.0.1:51342)
0619 16:57:42.387853 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:42.387862 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:42.387900 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:42.388366 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:42.388367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:42.388369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:42.388420 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.388423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.389270 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.389271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.390583 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.390586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.391531 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.391532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.391615 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.391618 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:42.391667 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 16:57:42.391787 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":32,"thread_start_us":49,"threads_started":1}
I20260619 16:57:42.392004 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.387632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.30.148.129:35691)
0619 16:57:42.387823 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:42.387827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:42.387984 (+ 157us) server_negotiation.cc:407] Connection header received
0619 16:57:42.388024 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:42.388026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:42.388052 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:42.388277 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:42.388467 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.389519 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.390795 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.391331 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.391393 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.391441 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:42.391869 (+ 428us) server_negotiation.cc:299] Negotiation successful
0619 16:57:42.391928 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":87,"threads_started":1}
I20260619 16:57:42.392104 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.387706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51342 (local address 127.30.148.130:40887)
0619 16:57:42.388036 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:42.388039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:42.388055 (+ 16us) server_negotiation.cc:407] Connection header received
0619 16:57:42.388083 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:42.388084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:42.388175 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:42.388251 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:42.388481 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.389159 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.390665 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.391381 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.391449 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.391496 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:42.391970 (+ 474us) server_negotiation.cc:299] Negotiation successful
0619 16:57:42.392016 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":81,"threads_started":1}
I20260619 16:57:42.392328 16213 tablet_service.cc:1467] Processing CreateTablet for tablet c194a459b52a4dd8b92855d7b4b1039e (DEFAULT_TABLE table=test-table [id=dc7b39ab83db48c192d84dc529decb6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:42.392632 16213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c194a459b52a4dd8b92855d7b4b1039e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:42.392645 16079 tablet_service.cc:1467] Processing CreateTablet for tablet c194a459b52a4dd8b92855d7b4b1039e (DEFAULT_TABLE table=test-table [id=dc7b39ab83db48c192d84dc529decb6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:42.392670 15945 tablet_service.cc:1467] Processing CreateTablet for tablet c194a459b52a4dd8b92855d7b4b1039e (DEFAULT_TABLE table=test-table [id=dc7b39ab83db48c192d84dc529decb6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:42.392863 16079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c194a459b52a4dd8b92855d7b4b1039e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:42.392948 15945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c194a459b52a4dd8b92855d7b4b1039e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:42.395380 16293 tablet_bootstrap.cc:492] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad: Bootstrap starting.
I20260619 16:57:42.395815 16294 tablet_bootstrap.cc:492] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5: Bootstrap starting.
I20260619 16:57:42.396063 16293 tablet_bootstrap.cc:654] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:42.396184 16295 tablet_bootstrap.cc:492] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd: Bootstrap starting.
I20260619 16:57:42.396477 16294 tablet_bootstrap.cc:654] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:42.396632 16293 log.cc:824] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:42.396787 16295 tablet_bootstrap.cc:654] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:42.397035 16294 log.cc:824] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:42.397310 16295 log.cc:824] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:42.397480 16293 tablet_bootstrap.cc:492] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad: No bootstrap required, opened a new log
I20260619 16:57:42.397557 16293 ts_tablet_manager.cc:1397] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:57:42.397809 16294 tablet_bootstrap.cc:492] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5: No bootstrap required, opened a new log
I20260619 16:57:42.397886 16294 ts_tablet_manager.cc:1397] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:57:42.397998 16295 tablet_bootstrap.cc:492] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd: No bootstrap required, opened a new log
I20260619 16:57:42.398067 16295 ts_tablet_manager.cc:1397] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 16:57:42.399363 16294 raft_consensus.cc:348] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 42427 } } peers { permanent_uuid: "bc0d0cb8a27c49a39335a247115cabad" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 40887 } } peers { permanent_uuid: "2f83a1982c264585a4428a90dfd160bd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 35691 } }
I20260619 16:57:42.399514 16294 raft_consensus.cc:374] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:42.399557 16294 raft_consensus.cc:729] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4000bb9ef3b4930bf4de1df7995edc5, State: Initialized, Role: FOLLOWER
I20260619 16:57:42.399490 16293 raft_consensus.cc:348] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 42427 } } peers { permanent_uuid: "bc0d0cb8a27c49a39335a247115cabad" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 40887 } } peers { permanent_uuid: "2f83a1982c264585a4428a90dfd160bd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 35691 } }
I20260619 16:57:42.399659 16293 raft_consensus.cc:374] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:42.399712 16293 raft_consensus.cc:729] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc0d0cb8a27c49a39335a247115cabad, State: Initialized, Role: FOLLOWER
I20260619 16:57:42.399680 16294 consensus_queue.cc:260] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 42427 } } peers { permanent_uuid: "bc0d0cb8a27c49a39335a247115cabad" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 40887 } } peers { permanent_uuid: "2f83a1982c264585a4428a90dfd160bd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 35691 } }
I20260619 16:57:42.399832 16293 consensus_queue.cc:260] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 42427 } } peers { permanent_uuid: "bc0d0cb8a27c49a39335a247115cabad" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 40887 } } peers { permanent_uuid: "2f83a1982c264585a4428a90dfd160bd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 35691 } }
I20260619 16:57:42.399945 16294 ts_tablet_manager.cc:1428] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:57:42.400002 16279 heartbeater.cc:503] Master 127.30.148.190:36263 was elected leader, sending a full tablet report...
I20260619 16:57:42.400100 16293 ts_tablet_manager.cc:1428] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 16:57:42.400210 16145 heartbeater.cc:503] Master 127.30.148.190:36263 was elected leader, sending a full tablet report...
I20260619 16:57:42.400264 16295 raft_consensus.cc:348] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 42427 } } peers { permanent_uuid: "bc0d0cb8a27c49a39335a247115cabad" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 40887 } } peers { permanent_uuid: "2f83a1982c264585a4428a90dfd160bd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 35691 } }
I20260619 16:57:42.400420 16295 raft_consensus.cc:374] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:42.400452 16295 raft_consensus.cc:729] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f83a1982c264585a4428a90dfd160bd, State: Initialized, Role: FOLLOWER
I20260619 16:57:42.400558 16295 consensus_queue.cc:260] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 42427 } } peers { permanent_uuid: "bc0d0cb8a27c49a39335a247115cabad" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 40887 } } peers { permanent_uuid: "2f83a1982c264585a4428a90dfd160bd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 35691 } }
I20260619 16:57:42.400858 16011 heartbeater.cc:503] Master 127.30.148.190:36263 was elected leader, sending a full tablet report...
I20260619 16:57:42.400934 16295 ts_tablet_manager.cc:1428] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260619 16:57:42.461308 16146 tablet.cc:2369] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:57:42.542474 16299 raft_consensus.cc:482] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:57:42.542595 16299 raft_consensus.cc:504] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 42427 } } peers { permanent_uuid: "bc0d0cb8a27c49a39335a247115cabad" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 40887 } } peers { permanent_uuid: "2f83a1982c264585a4428a90dfd160bd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 35691 } }
I20260619 16:57:42.542999 16299 leader_election.cc:288] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc0d0cb8a27c49a39335a247115cabad (127.30.148.130:40887), 2f83a1982c264585a4428a90dfd160bd (127.30.148.129:35691)
I20260619 16:57:42.546870 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.543210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:41627 (local address 127.30.148.130:40887)
0619 16:57:42.543266 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:42.543269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:42.543280 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:57:42.543370 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:42.543371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:42.543381 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:42.543440 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:42.543611 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.544373 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.545920 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.546476 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.546556 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.546565 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:42.546758 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 16:57:42.546798 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:57:42.546870 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.543216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:35327 (local address 127.30.148.129:35691)
0619 16:57:42.543262 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:42.543264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:42.543492 (+ 228us) server_negotiation.cc:407] Connection header received
0619 16:57:42.543526 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:42.543527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:42.543534 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:42.543589 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:42.543707 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.544397 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.545924 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.546477 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.546556 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.546565 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:42.546767 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 16:57:42.546805 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:57:42.546922 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.543181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:40887 (local address 127.30.148.131:41627)
0619 16:57:42.543231 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:42.543243 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:42.543291 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:42.543464 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:42.543465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:42.543467 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:42.543543 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.543545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.544484 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.544486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.545833 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.545837 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.546580 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.546581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.546675 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.546677 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:57:42.546745 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 16:57:42.546800 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":52}
I20260619 16:57:42.546934 16302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.543183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:35691 (local address 127.30.148.131:35327)
0619 16:57:42.543415 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:42.543424 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:42.543479 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:42.543581 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:42.543582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:42.543583 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:42.543642 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.543643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.544484 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.544486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.545833 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.545837 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.546580 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.546581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.546671 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.546676 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:57:42.546745 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 16:57:42.546800 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":110,"thread_start_us":113,"threads_started":1}
I20260619 16:57:42.547286 16099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c194a459b52a4dd8b92855d7b4b1039e" candidate_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc0d0cb8a27c49a39335a247115cabad" is_pre_election: true
I20260619 16:57:42.547451 16099 raft_consensus.cc:2454] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4000bb9ef3b4930bf4de1df7995edc5 in term 0.
I20260619 16:57:42.547690 16168 leader_election.cc:302] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4000bb9ef3b4930bf4de1df7995edc5, bc0d0cb8a27c49a39335a247115cabad; no voters:
I20260619 16:57:42.547740 15965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c194a459b52a4dd8b92855d7b4b1039e" candidate_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f83a1982c264585a4428a90dfd160bd" is_pre_election: true
I20260619 16:57:42.547827 16299 raft_consensus.cc:2781] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:57:42.547892 16299 raft_consensus.cc:482] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:57:42.547900 15965 raft_consensus.cc:2454] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4000bb9ef3b4930bf4de1df7995edc5 in term 0.
I20260619 16:57:42.547922 16299 raft_consensus.cc:3037] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:42.548646 16299 raft_consensus.cc:504] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 42427 } } peers { permanent_uuid: "bc0d0cb8a27c49a39335a247115cabad" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 40887 } } peers { permanent_uuid: "2f83a1982c264585a4428a90dfd160bd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 35691 } }
I20260619 16:57:42.548795 16299 leader_election.cc:288] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [CANDIDATE]: Term 1 election: Requested vote from peers bc0d0cb8a27c49a39335a247115cabad (127.30.148.130:40887), 2f83a1982c264585a4428a90dfd160bd (127.30.148.129:35691)
I20260619 16:57:42.548955 15965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c194a459b52a4dd8b92855d7b4b1039e" candidate_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f83a1982c264585a4428a90dfd160bd"
I20260619 16:57:42.548974 16099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c194a459b52a4dd8b92855d7b4b1039e" candidate_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc0d0cb8a27c49a39335a247115cabad"
I20260619 16:57:42.549028 16099 raft_consensus.cc:3037] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:42.549038 15965 raft_consensus.cc:3037] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:42.549762 16099 raft_consensus.cc:2454] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4000bb9ef3b4930bf4de1df7995edc5 in term 1.
I20260619 16:57:42.549852 15965 raft_consensus.cc:2454] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4000bb9ef3b4930bf4de1df7995edc5 in term 1.
I20260619 16:57:42.549911 16168 leader_election.cc:302] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4000bb9ef3b4930bf4de1df7995edc5, bc0d0cb8a27c49a39335a247115cabad; no voters:
I20260619 16:57:42.550030 16299 raft_consensus.cc:2781] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:42.550163 16299 raft_consensus.cc:686] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 1 LEADER]: Becoming Leader. State: Replica: a4000bb9ef3b4930bf4de1df7995edc5, State: Running, Role: LEADER
I20260619 16:57:42.550251 16299 consensus_queue.cc:237] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 42427 } } peers { permanent_uuid: "bc0d0cb8a27c49a39335a247115cabad" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 40887 } } peers { permanent_uuid: "2f83a1982c264585a4428a90dfd160bd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 35691 } }
I20260619 16:57:42.551024 15819 catalog_manager.cc:5495] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4000bb9ef3b4930bf4de1df7995edc5 (127.30.148.131). New cstate: current_term: 1 leader_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4000bb9ef3b4930bf4de1df7995edc5" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 42427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc0d0cb8a27c49a39335a247115cabad" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f83a1982c264585a4428a90dfd160bd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 35691 } health_report { overall_health: UNKNOWN } } }
W20260619 16:57:42.557536 16012 tablet.cc:2369] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:57:42.572780 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.570739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51752 (local address 127.30.148.131:42427)
0619 16:57:42.570777 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:42.570779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:42.570811 (+ 32us) server_negotiation.cc:407] Connection header received
0619 16:57:42.570877 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:42.570877 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:42.570892 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:42.570948 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:57:42.571054 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.571499 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.572197 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.572367 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.572413 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.572529 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:57:42.572619 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:57:42.572671 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:42.572714 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 16:57:42.572730 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:57:42.574486 15965 raft_consensus.cc:1261] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd [term 1 FOLLOWER]: Refusing update from remote peer a4000bb9ef3b4930bf4de1df7995edc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:42.574597 16099 raft_consensus.cc:1261] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad [term 1 FOLLOWER]: Refusing update from remote peer a4000bb9ef3b4930bf4de1df7995edc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:42.574790 16299 consensus_queue.cc:1035] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f83a1982c264585a4428a90dfd160bd" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 35691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:42.574937 16303 consensus_queue.cc:1035] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc0d0cb8a27c49a39335a247115cabad" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:42.577519 16306 mvcc.cc:205] Tried to move back new op lower bound from 7298614323502465024 to 7298614323406417920. Current Snapshot: MvccSnapshot[applied={T|T < 7298614323502465024}]
I20260619 16:57:42.577783 16308 mvcc.cc:205] Tried to move back new op lower bound from 7298614323502465024 to 7298614323406417920. Current Snapshot: MvccSnapshot[applied={T|T < 7298614323502465024}]
I20260619 16:57:42.578569 16307 mvcc.cc:205] Tried to move back new op lower bound from 7298614323502465024 to 7298614323406417920. Current Snapshot: MvccSnapshot[applied={T|T < 7298614323502465024 or (T in {7298614323502465024})}]
I20260619 16:57:42.579667 15819 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:42.579764 15819 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:42.580462 15819 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92afb10cb21145e19692f7f9b435ed7a: Sending DeleteTablet for 3 replicas of tablet c194a459b52a4dd8b92855d7b4b1039e
I20260619 16:57:42.580770 16079 tablet_service.cc:1514] Processing DeleteTablet for tablet c194a459b52a4dd8b92855d7b4b1039e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:42 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:51342
I20260619 16:57:42.580760 15945 tablet_service.cc:1514] Processing DeleteTablet for tablet c194a459b52a4dd8b92855d7b4b1039e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:42 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:33698
I20260619 16:57:42.580984 16317 tablet_replica.cc:330] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd: stopping tablet replica
I20260619 16:57:42.580986 16213 tablet_service.cc:1514] Processing DeleteTablet for tablet c194a459b52a4dd8b92855d7b4b1039e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:42 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:51750
I20260619 16:57:42.581038 16316 tablet_replica.cc:330] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad: stopping tablet replica
I20260619 16:57:42.581080 16317 raft_consensus.cc:2229] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:42.581152 16316 raft_consensus.cc:2229] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:42.581223 16317 raft_consensus.cc:2258] T c194a459b52a4dd8b92855d7b4b1039e P 2f83a1982c264585a4428a90dfd160bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:42.581243 16316 raft_consensus.cc:2258] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:42.581398 16318 tablet_replica.cc:330] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5: stopping tablet replica
I20260619 16:57:42.581514 16318 raft_consensus.cc:2229] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:57:42.581637 16318 raft_consensus.cc:2258] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:42.581634 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 15882
I20260619 16:57:42.581687 16316 ts_tablet_manager.cc:1905] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:42.583274 16316 ts_tablet_manager.cc:1918] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:42.583331 16316 log.cc:1196] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/c194a459b52a4dd8b92855d7b4b1039e
I20260619 16:57:42.583603 16316 ts_tablet_manager.cc:1939] T c194a459b52a4dd8b92855d7b4b1039e P bc0d0cb8a27c49a39335a247115cabad: Deleting consensus metadata
I20260619 16:57:42.583993 15806 catalog_manager.cc:4841] TS bc0d0cb8a27c49a39335a247115cabad (127.30.148.130:40887): tablet c194a459b52a4dd8b92855d7b4b1039e (table test-table [id=dc7b39ab83db48c192d84dc529decb6f]) successfully deleted
I20260619 16:57:42.584956 16318 ts_tablet_manager.cc:1905] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:42.586975 16318 ts_tablet_manager.cc:1918] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:42.587136 16318 log.cc:1196] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/c194a459b52a4dd8b92855d7b4b1039e
W20260619 16:57:42.587235 15804 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:35691 (error 108)
I20260619 16:57:42.587507 16318 ts_tablet_manager.cc:1939] T c194a459b52a4dd8b92855d7b4b1039e P a4000bb9ef3b4930bf4de1df7995edc5: Deleting consensus metadata
I20260619 16:57:42.587663 16287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.587550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:35691 (local address 127.0.0.1:33700)
0619 16:57:42.587589 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:42.587627 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:35691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260619 16:57:42.587800 15804 catalog_manager.cc:4584] TS 2f83a1982c264585a4428a90dfd160bd (127.30.148.129:35691): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c194a459b52a4dd8b92855d7b4b1039e: Network error: Client connection negotiation failed: client connection to 127.30.148.129:35691: connect: Connection refused (error 111)
I20260619 16:57:42.587894 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 16016
I20260619 16:57:42.588508 15805 catalog_manager.cc:4841] TS a4000bb9ef3b4930bf4de1df7995edc5 (127.30.148.131:42427): tablet c194a459b52a4dd8b92855d7b4b1039e (table test-table [id=dc7b39ab83db48c192d84dc529decb6f]) successfully deleted
I20260619 16:57:42.592927 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 16150
I20260619 16:57:42.597443 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 15788
2026-06-19T16:57:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1990 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T16:57:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:42Z Disabled control of system clock
I20260619 16:57:42.622941 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:32977
--webserver_interface=127.30.148.190
--webserver_port=0
--builtin_ntp_servers=127.30.148.148:35093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:32977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 16:57:42.702518 16323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:42.702708 16323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:42.702728 16323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:42.703812 16323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:42.703850 16323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:42.703861 16323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:42.703874 16323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:42.703922 16323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:42.705229 16323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:32977
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:32977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:42.705475 16323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:42.705736 16323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:42.710511 16331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:42.710575 16323 server_base.cc:1034] running on GCE node
W20260619 16:57:42.710525 16328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:42.710526 16329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:42.710986 16323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:42.711277 16323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:42.712419 16323 hybrid_clock.cc:648] HybridClock initialized: now 1781888262712402 us; error 34 us; skew 500 ppm
I20260619 16:57:42.716305 16323 webserver.cc:460] Webserver started at http://127.30.148.190:45523/ using document root <none> and password file <none>
I20260619 16:57:42.716548 16323 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:42.716621 16323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:42.716763 16323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:42.717839 16323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "df3ff796c9f142d3a4dcdc00386cb912"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "6c516d5f712f2c717a64641a1358ec17"
server_key_iv: "8b24b955600889115d60649c643ebdb2"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.718182 16323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "df3ff796c9f142d3a4dcdc00386cb912"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "6c516d5f712f2c717a64641a1358ec17"
server_key_iv: "8b24b955600889115d60649c643ebdb2"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.719398 16323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:57:42.720127 16337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.720283 16323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:42.720376 16323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "df3ff796c9f142d3a4dcdc00386cb912"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "6c516d5f712f2c717a64641a1358ec17"
server_key_iv: "8b24b955600889115d60649c643ebdb2"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.720443 16323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:42.743234 16323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:42.743489 16323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:42.743626 16323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:42.747494 16323 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:32977
I20260619 16:57:42.747524 16389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:32977 every 8 connection(s)
I20260619 16:57:42.747844 16323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:42.748744 16390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:42.750873 16390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912: Bootstrap starting.
I20260619 16:57:42.751382 16390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:42.751619 16390 log.cc:824] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:42.752239 16390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912: No bootstrap required, opened a new log
I20260619 16:57:42.753798 16390 raft_consensus.cc:348] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3ff796c9f142d3a4dcdc00386cb912" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32977 } }
I20260619 16:57:42.753919 16390 raft_consensus.cc:374] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:42.753940 16390 raft_consensus.cc:729] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df3ff796c9f142d3a4dcdc00386cb912, State: Initialized, Role: FOLLOWER
I20260619 16:57:42.754077 16390 consensus_queue.cc:260] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3ff796c9f142d3a4dcdc00386cb912" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32977 } }
I20260619 16:57:42.754158 16390 raft_consensus.cc:388] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:42.754187 16390 raft_consensus.cc:482] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:42.754220 16390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:42.754709 16390 raft_consensus.cc:504] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3ff796c9f142d3a4dcdc00386cb912" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32977 } }
I20260619 16:57:42.754787 16390 leader_election.cc:302] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df3ff796c9f142d3a4dcdc00386cb912; no voters:
I20260619 16:57:42.754901 16390 leader_election.cc:288] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:42.754976 16393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:42.755167 16393 raft_consensus.cc:686] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [term 1 LEADER]: Becoming Leader. State: Replica: df3ff796c9f142d3a4dcdc00386cb912, State: Running, Role: LEADER
I20260619 16:57:42.755173 16390 sys_catalog.cc:564] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:42.755283 16393 consensus_queue.cc:237] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3ff796c9f142d3a4dcdc00386cb912" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32977 } }
I20260619 16:57:42.755636 16394 sys_catalog.cc:455] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df3ff796c9f142d3a4dcdc00386cb912" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3ff796c9f142d3a4dcdc00386cb912" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32977 } } }
I20260619 16:57:42.755714 16395 sys_catalog.cc:455] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df3ff796c9f142d3a4dcdc00386cb912. Latest consensus state: current_term: 1 leader_uuid: "df3ff796c9f142d3a4dcdc00386cb912" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3ff796c9f142d3a4dcdc00386cb912" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 32977 } } }
I20260619 16:57:42.755731 16394 sys_catalog.cc:458] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:42.755858 16395 sys_catalog.cc:458] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:42.756199 16402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:42.756722 16402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:42.756903 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 16323
I20260619 16:57:42.756992 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:42.757193 31314 external_mini_cluster.cc:1428] Setting key 467b47755b05065b504e4e303972c63d
I20260619 16:57:42.758211 16402 catalog_manager.cc:1347] Generated new cluster ID: d1c4c2b0c553474cb40f1fa2fbbc0d8f
I20260619 16:57:42.758261 16402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:42.762028 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.757763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57848 (local address 127.30.148.190:32977)
0619 16:57:42.757958 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:42.757968 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:42.758001 (+ 33us) server_negotiation.cc:407] Connection header received
0619 16:57:42.758105 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:42.758111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:42.758177 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:42.758312 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:42.758608 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.759427 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.760193 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.760580 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.760703 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.761344 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:42.761347 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:42.761348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:42.761532 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:42.761562 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:42.761591 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:42.761680 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:42.761753 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 16:57:42.761825 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260619 16:57:42.771080 16402 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:42.771629 16402 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:42.775696 16402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912: Generated new TSK 0
I20260619 16:57:42.775857 16402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:57:42.780308 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:32977
--builtin_ntp_servers=127.30.148.148:35093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 16:57:42.860005 16414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:42.860195 16414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:42.860235 16414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:42.861327 16414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:42.861413 16414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:42.861452 16414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:57:42.862789 16414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:32977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:42.863006 16414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:42.863299 16414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:42.868081 16419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:42.868256 16420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:42.868289 16414 server_base.cc:1034] running on GCE node
W20260619 16:57:42.868081 16422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:42.868520 16414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:42.868768 16414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:42.869920 16414 hybrid_clock.cc:648] HybridClock initialized: now 1781888262869907 us; error 28 us; skew 500 ppm
I20260619 16:57:42.871187 16414 webserver.cc:460] Webserver started at http://127.30.148.129:34227/ using document root <none> and password file <none>
I20260619 16:57:42.871420 16414 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:42.871481 16414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:42.871625 16414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:42.872617 16414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "37e5374952d345ae98007a6f7ca8c667"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "2160f1931d53b16f7369bdd339221a35"
server_key_iv: "8055e6a561a29e3dc4513c493e4b5c95"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.872967 16414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "37e5374952d345ae98007a6f7ca8c667"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "2160f1931d53b16f7369bdd339221a35"
server_key_iv: "8055e6a561a29e3dc4513c493e4b5c95"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.874325 16414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:57:42.875070 16428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.875280 16414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:57:42.875375 16414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "37e5374952d345ae98007a6f7ca8c667"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "2160f1931d53b16f7369bdd339221a35"
server_key_iv: "8055e6a561a29e3dc4513c493e4b5c95"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.875442 16414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:42.892278 16414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:42.892498 16414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:42.892593 16414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:42.892750 16414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:42.892984 16414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:42.893013 16414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.893033 16414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:42.893052 16414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:42.898535 16414 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:36135
I20260619 16:57:42.898609 16541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:36135 every 8 connection(s)
I20260619 16:57:42.898909 16414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:57:42.903591 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.899559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:45101 (local address 127.30.148.190:32977)
0619 16:57:42.899601 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:42.899603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:42.899851 (+ 248us) server_negotiation.cc:407] Connection header received
0619 16:57:42.900293 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:42.900294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:42.900304 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:42.900354 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:42.900931 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.901457 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.902372 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.902578 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.902643 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.903374 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:42.903375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:42.903376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:42.903379 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:42.903393 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:42.903396 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:42.903449 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:42.903511 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 16:57:42.903537 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:57:42.903789 16544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:42.899546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32977 (local address 127.30.148.129:45101)
0619 16:57:42.899705 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:42.899753 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:42.900214 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:42.900401 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:42.900404 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:42.900412 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:42.900867 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.900870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.901524 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.901525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.902278 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:42.902280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:42.902656 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:42.902657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:42.902757 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:42.903248 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:57:42.903249 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:42.903274 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:57:42.903295 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:42.903451 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:42.903453 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:42.903455 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:57:42.903501 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 16:57:42.903571 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":85,"threads_started":1}
I20260619 16:57:42.904278 16542 heartbeater.cc:347] Connected to a master server at 127.30.148.190:32977
I20260619 16:57:42.904369 16542 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:42.904536 16542 heartbeater.cc:511] Master 127.30.148.190:32977 requested a full tablet report, sending...
I20260619 16:57:42.904764 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 16414
I20260619 16:57:42.904879 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:57:42.905048 16354 ts_manager.cc:194] Registered new tserver with Master: 37e5374952d345ae98007a6f7ca8c667 (127.30.148.129:36135)
I20260619 16:57:42.905086 31314 external_mini_cluster.cc:1428] Setting key 0b4adbb937799b45594397f91308301f
I20260619 16:57:42.905730 16354 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.148.129:45101
I20260619 16:57:42.906179 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:32977
--builtin_ntp_servers=127.30.148.148:35093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 16:57:42.984738 16545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:42.984947 16545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:42.984983 16545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:42.986186 16545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:42.986245 16545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:42.986285 16545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:57:42.987602 16545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:32977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:42.987845 16545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:42.988139 16545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:42.992995 16553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:42.993000 16550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:42.993000 16551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:42.993275 16545 server_base.cc:1034] running on GCE node
I20260619 16:57:42.993512 16545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:42.993762 16545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:42.994917 16545 hybrid_clock.cc:648] HybridClock initialized: now 1781888262994902 us; error 27 us; skew 500 ppm
I20260619 16:57:42.996104 16545 webserver.cc:460] Webserver started at http://127.30.148.130:37495/ using document root <none> and password file <none>
I20260619 16:57:42.996335 16545 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:42.996399 16545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:42.996529 16545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:42.997596 16545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "7dbaab16898f492ebf5745538e34e704"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "a2cdb9f07c4ff30bb8c1da381586fc61"
server_key_iv: "2359ae1aebb99f088f69a138ab606dd4"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.997934 16545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "7dbaab16898f492ebf5745538e34e704"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "a2cdb9f07c4ff30bb8c1da381586fc61"
server_key_iv: "2359ae1aebb99f088f69a138ab606dd4"
server_key_version: "encryptionkey@0"
I20260619 16:57:42.999191 16545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:42.999871 16559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:43.000034 16545 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:43.000128 16545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "7dbaab16898f492ebf5745538e34e704"
format_stamp: "Formatted at 2026-06-19 16:57:42 on dist-test-slave-xb9n"
server_key: "a2cdb9f07c4ff30bb8c1da381586fc61"
server_key_iv: "2359ae1aebb99f088f69a138ab606dd4"
server_key_version: "encryptionkey@0"
I20260619 16:57:43.000203 16545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:43.011799 16545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:43.012059 16545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:43.012208 16545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:43.012421 16545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:43.012718 16545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:43.012765 16545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:43.012797 16545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:43.012825 16545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:43.018530 16545 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:43395
I20260619 16:57:43.018585 16672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:43395 every 8 connection(s)
I20260619 16:57:43.018870 16545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:57:43.020551 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 16545
I20260619 16:57:43.020673 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:57:43.020861 31314 external_mini_cluster.cc:1428] Setting key 88e793da5665d92192ebf0123facd64b
I20260619 16:57:43.021688 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:32977
--builtin_ntp_servers=127.30.148.148:35093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 16:57:43.024437 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.019507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:45463 (local address 127.30.148.190:32977)
0619 16:57:43.019579 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:43.019582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:43.019768 (+ 186us) server_negotiation.cc:407] Connection header received
0619 16:57:43.020211 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:43.020212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:43.020222 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:43.020267 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:43.020957 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.021613 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.022761 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.023059 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.023136 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.024146 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:43.024148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:43.024148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:43.024152 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:43.024166 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:43.024170 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:43.024247 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:43.024334 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 16:57:43.024362 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 16:57:43.024643 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.019482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32977 (local address 127.30.148.130:45463)
0619 16:57:43.019670 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:43.019698 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:43.020130 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:43.020322 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:43.020325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:43.020332 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:43.020862 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.020867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.021703 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.021704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.022683 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.022685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.023139 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.023140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.023328 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.023958 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:57:43.023958 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:43.024010 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:57:43.024035 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:43.024248 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:43.024254 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:43.024257 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:43.024338 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 16:57:43.024434 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260619 16:57:43.025048 16673 heartbeater.cc:347] Connected to a master server at 127.30.148.190:32977
I20260619 16:57:43.025153 16673 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:43.025349 16673 heartbeater.cc:511] Master 127.30.148.190:32977 requested a full tablet report, sending...
I20260619 16:57:43.025717 16354 ts_manager.cc:194] Registered new tserver with Master: 7dbaab16898f492ebf5745538e34e704 (127.30.148.130:43395)
I20260619 16:57:43.026078 16354 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.148.130:45463
W20260619 16:57:43.101405 16676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:43.101588 16676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:43.101608 16676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:43.102715 16676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:43.102749 16676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:43.102773 16676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:57:43.104153 16676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:32977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:43.104326 16676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:43.104547 16676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:43.109252 16684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:43.109238 16681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:43.109349 16676 server_base.cc:1034] running on GCE node
W20260619 16:57:43.109261 16682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:43.109659 16676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:43.109838 16676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:43.110977 16676 hybrid_clock.cc:648] HybridClock initialized: now 1781888263110965 us; error 28 us; skew 500 ppm
I20260619 16:57:43.112123 16676 webserver.cc:460] Webserver started at http://127.30.148.131:45153/ using document root <none> and password file <none>
I20260619 16:57:43.112330 16676 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:43.112370 16676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:43.112515 16676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:43.113579 16676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "56add77b2187447394fb3e3e61c3d066"
format_stamp: "Formatted at 2026-06-19 16:57:43 on dist-test-slave-xb9n"
server_key: "9d482cab7e37b6941a434fc2e52c5d32"
server_key_iv: "c021a41d526f2af86c4b3cd9e3ba8d92"
server_key_version: "encryptionkey@0"
I20260619 16:57:43.113894 16676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "56add77b2187447394fb3e3e61c3d066"
format_stamp: "Formatted at 2026-06-19 16:57:43 on dist-test-slave-xb9n"
server_key: "9d482cab7e37b6941a434fc2e52c5d32"
server_key_iv: "c021a41d526f2af86c4b3cd9e3ba8d92"
server_key_version: "encryptionkey@0"
I20260619 16:57:43.115026 16676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:43.115702 16690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:43.115844 16676 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:43.115918 16676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "56add77b2187447394fb3e3e61c3d066"
format_stamp: "Formatted at 2026-06-19 16:57:43 on dist-test-slave-xb9n"
server_key: "9d482cab7e37b6941a434fc2e52c5d32"
server_key_iv: "c021a41d526f2af86c4b3cd9e3ba8d92"
server_key_version: "encryptionkey@0"
I20260619 16:57:43.115968 16676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:43.142944 16676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:43.143173 16676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:43.143268 16676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:43.143420 16676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:43.143669 16676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:43.143703 16676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:43.143723 16676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:43.143735 16676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:43.149181 16676 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:34523
I20260619 16:57:43.149236 16803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:34523 every 8 connection(s)
I20260619 16:57:43.149542 16676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:57:43.154116 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.150224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:40383 (local address 127.30.148.190:32977)
0619 16:57:43.150274 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:43.150276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:43.150491 (+ 215us) server_negotiation.cc:407] Connection header received
0619 16:57:43.150935 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:43.150936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:43.150946 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:43.150987 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:43.151570 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.152043 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.152963 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.153160 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.153213 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.153899 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:43.153900 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:43.153901 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:43.153904 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:43.153916 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:43.153920 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:43.153962 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:43.154024 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 16:57:43.154047 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:57:43.154320 16806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.150220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:32977 (local address 127.30.148.131:40383)
0619 16:57:43.150387 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:43.150416 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:43.150847 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:43.151035 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:43.151038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:43.151046 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:43.151505 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.151508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.152131 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.152132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.152889 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.152891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.153228 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.153228 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.153336 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.153785 (+ 449us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:57:43.153785 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:43.153810 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:57:43.153830 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:43.153971 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:43.153975 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:43.153977 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:57:43.154027 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 16:57:43.154095 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260619 16:57:43.154808 16804 heartbeater.cc:347] Connected to a master server at 127.30.148.190:32977
I20260619 16:57:43.154881 16804 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:43.155090 16804 heartbeater.cc:511] Master 127.30.148.190:32977 requested a full tablet report, sending...
I20260619 16:57:43.155429 16354 ts_manager.cc:194] Registered new tserver with Master: 56add77b2187447394fb3e3e61c3d066 (127.30.148.131:34523)
I20260619 16:57:43.155777 16354 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.148.131:40383
I20260619 16:57:43.156725 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 16676
I20260619 16:57:43.156800 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:57:43.156955 31314 external_mini_cluster.cc:1428] Setting key b7620681541d9cbe306965e8cf067718
I20260619 16:57:43.157768 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:57:43.161290 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.159022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57858 (local address 127.30.148.190:32977)
0619 16:57:43.159057 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:43.159058 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:43.159146 (+ 88us) server_negotiation.cc:407] Connection header received
0619 16:57:43.159218 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:43.159219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:43.159227 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:43.159265 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:43.159394 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.159812 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.160455 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.160612 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.160657 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.161136 (+ 479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:43.161137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:43.161137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:43.161140 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:43.161150 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:43.161153 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:43.161196 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:43.161229 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 16:57:43.161246 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 16:57:43.162617 16354 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:57:43.164165 16354 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:57:43.169997 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.166212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43780 (local address 127.30.148.130:43395)
0619 16:57:43.166387 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:43.166392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:43.166403 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:57:43.166442 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:43.166444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:43.166460 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:43.166543 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:43.166715 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.167528 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.168929 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.169620 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.169708 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.169790 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:43.169879 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 16:57:43.169914 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260619 16:57:43.169997 16815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.166266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:34523 (local address 127.0.0.1:57228)
0619 16:57:43.166402 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:43.166410 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:43.166442 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:43.166943 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:43.166944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:43.166945 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:43.166977 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.166978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.168209 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.168210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.168918 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.168920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.169738 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.169740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.169801 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.169806 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:57:43.169836 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 16:57:43.169889 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":69,"threads_started":1}
I20260619 16:57:43.170008 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.166016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:43395 (local address 127.0.0.1:43780)
0619 16:57:43.166133 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:43.166172 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:43.166233 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:43.166562 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:43.166565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:43.166567 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:43.166627 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.166628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.167612 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.167613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.168608 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.168610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.169706 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.169707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.169780 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.169792 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 16:57:43.169867 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 16:57:43.169911 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":69,"threads_started":1}
I20260619 16:57:43.170401 16817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.166627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57228 (local address 127.30.148.131:34523)
0619 16:57:43.166788 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:43.166792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:43.166801 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:57:43.166831 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:43.166833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:43.166847 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:43.167041 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:43.167082 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.168105 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.169016 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.169627 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.170176 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.170250 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:43.170272 (+ 22us) server_negotiation.cc:299] Negotiation successful
0619 16:57:43.170305 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260619 16:57:43.170563 16813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.166053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36135 (local address 127.0.0.1:55126)
0619 16:57:43.166228 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:43.166239 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:43.166293 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:43.166816 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:43.166817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:43.166819 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:43.166853 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.166854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.168157 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.168158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.169224 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.169227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.170248 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.170249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.170310 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.170313 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:43.170447 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 16:57:43.170492 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":20,"thread_start_us":114,"threads_started":1}
I20260619 16:57:43.170823 16816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.166411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55126 (local address 127.30.148.129:36135)
0619 16:57:43.166583 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:43.166587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:43.166596 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:57:43.166633 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:43.166636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:43.166651 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:43.166726 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:43.166934 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.168069 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.169312 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.170051 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.170120 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.170214 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:43.170704 (+ 490us) server_negotiation.cc:299] Negotiation successful
0619 16:57:43.170746 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260619 16:57:43.170986 16738 tablet_service.cc:1467] Processing CreateTablet for tablet 34d29c62b2e54f7fbcddc8c02be77a16 (DEFAULT_TABLE table=test-table [id=039902410fb74b29b8ff5a4c3ddad9e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:43.170982 16607 tablet_service.cc:1467] Processing CreateTablet for tablet 34d29c62b2e54f7fbcddc8c02be77a16 (DEFAULT_TABLE table=test-table [id=039902410fb74b29b8ff5a4c3ddad9e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:43.171279 16738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34d29c62b2e54f7fbcddc8c02be77a16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:43.171309 16607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34d29c62b2e54f7fbcddc8c02be77a16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:43.171380 16476 tablet_service.cc:1467] Processing CreateTablet for tablet 34d29c62b2e54f7fbcddc8c02be77a16 (DEFAULT_TABLE table=test-table [id=039902410fb74b29b8ff5a4c3ddad9e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:43.171665 16476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34d29c62b2e54f7fbcddc8c02be77a16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:43.173493 16818 tablet_bootstrap.cc:492] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066: Bootstrap starting.
I20260619 16:57:43.174166 16818 tablet_bootstrap.cc:654] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:43.174383 16818 log.cc:824] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:43.175032 16818 tablet_bootstrap.cc:492] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066: No bootstrap required, opened a new log
I20260619 16:57:43.175088 16818 ts_tablet_manager.cc:1397] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:57:43.176787 16818 raft_consensus.cc:348] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dbaab16898f492ebf5745538e34e704" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 43395 } } peers { permanent_uuid: "37e5374952d345ae98007a6f7ca8c667" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36135 } } peers { permanent_uuid: "56add77b2187447394fb3e3e61c3d066" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34523 } }
I20260619 16:57:43.176913 16818 raft_consensus.cc:374] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:43.176934 16818 raft_consensus.cc:729] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56add77b2187447394fb3e3e61c3d066, State: Initialized, Role: FOLLOWER
I20260619 16:57:43.177069 16818 consensus_queue.cc:260] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dbaab16898f492ebf5745538e34e704" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 43395 } } peers { permanent_uuid: "37e5374952d345ae98007a6f7ca8c667" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36135 } } peers { permanent_uuid: "56add77b2187447394fb3e3e61c3d066" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34523 } }
I20260619 16:57:43.177304 16818 ts_tablet_manager.cc:1428] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:57:43.177528 16804 heartbeater.cc:503] Master 127.30.148.190:32977 was elected leader, sending a full tablet report...
I20260619 16:57:43.177521 16821 tablet_bootstrap.cc:492] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704: Bootstrap starting.
I20260619 16:57:43.178084 16821 tablet_bootstrap.cc:654] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:43.178395 16821 log.cc:824] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:43.178695 16820 tablet_bootstrap.cc:492] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667: Bootstrap starting.
I20260619 16:57:43.179092 16821 tablet_bootstrap.cc:492] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704: No bootstrap required, opened a new log
I20260619 16:57:43.179143 16821 ts_tablet_manager.cc:1397] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:57:43.179267 16820 tablet_bootstrap.cc:654] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:43.179551 16820 log.cc:824] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:43.180233 16820 tablet_bootstrap.cc:492] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667: No bootstrap required, opened a new log
I20260619 16:57:43.180301 16820 ts_tablet_manager.cc:1397] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:57:43.181120 16821 raft_consensus.cc:348] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dbaab16898f492ebf5745538e34e704" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 43395 } } peers { permanent_uuid: "37e5374952d345ae98007a6f7ca8c667" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36135 } } peers { permanent_uuid: "56add77b2187447394fb3e3e61c3d066" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34523 } }
I20260619 16:57:43.181239 16821 raft_consensus.cc:374] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:43.181260 16821 raft_consensus.cc:729] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dbaab16898f492ebf5745538e34e704, State: Initialized, Role: FOLLOWER
I20260619 16:57:43.181401 16821 consensus_queue.cc:260] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dbaab16898f492ebf5745538e34e704" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 43395 } } peers { permanent_uuid: "37e5374952d345ae98007a6f7ca8c667" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36135 } } peers { permanent_uuid: "56add77b2187447394fb3e3e61c3d066" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34523 } }
I20260619 16:57:43.181651 16821 ts_tablet_manager.cc:1428] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 16:57:43.181705 16673 heartbeater.cc:503] Master 127.30.148.190:32977 was elected leader, sending a full tablet report...
I20260619 16:57:43.182345 16820 raft_consensus.cc:348] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dbaab16898f492ebf5745538e34e704" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 43395 } } peers { permanent_uuid: "37e5374952d345ae98007a6f7ca8c667" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36135 } } peers { permanent_uuid: "56add77b2187447394fb3e3e61c3d066" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34523 } }
I20260619 16:57:43.182480 16820 raft_consensus.cc:374] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:43.182502 16820 raft_consensus.cc:729] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e5374952d345ae98007a6f7ca8c667, State: Initialized, Role: FOLLOWER
I20260619 16:57:43.182614 16820 consensus_queue.cc:260] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dbaab16898f492ebf5745538e34e704" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 43395 } } peers { permanent_uuid: "37e5374952d345ae98007a6f7ca8c667" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36135 } } peers { permanent_uuid: "56add77b2187447394fb3e3e61c3d066" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34523 } }
I20260619 16:57:43.182845 16820 ts_tablet_manager.cc:1428] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 16:57:43.182917 16542 heartbeater.cc:503] Master 127.30.148.190:32977 was elected leader, sending a full tablet report...
W20260619 16:57:43.269451 16674 tablet.cc:2369] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:57:43.365188 16826 raft_consensus.cc:482] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:57:43.365326 16826 raft_consensus.cc:504] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dbaab16898f492ebf5745538e34e704" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 43395 } } peers { permanent_uuid: "37e5374952d345ae98007a6f7ca8c667" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36135 } } peers { permanent_uuid: "56add77b2187447394fb3e3e61c3d066" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34523 } }
I20260619 16:57:43.365718 16826 leader_election.cc:288] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dbaab16898f492ebf5745538e34e704 (127.30.148.130:43395), 56add77b2187447394fb3e3e61c3d066 (127.30.148.131:34523)
I20260619 16:57:43.368502 16817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.366058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:41681 (local address 127.30.148.131:34523)
0619 16:57:43.366108 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:43.366110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:43.366120 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:57:43.366151 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:43.366152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:43.366161 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:43.366233 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:43.366394 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.366858 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.367769 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.368165 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.368244 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.368254 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:43.368386 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 16:57:43.368428 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 16:57:43.368505 16544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.365873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:34523 (local address 127.30.148.129:41681)
0619 16:57:43.365912 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:43.365923 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:43.365968 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:43.366232 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:43.366233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:43.366234 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:43.366315 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.366321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.366931 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.366932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.367687 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.367689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.368246 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.368247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.368315 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.368319 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:57:43.368368 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 16:57:43.368420 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":1}
I20260619 16:57:43.368526 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.365985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:38443 (local address 127.30.148.130:43395)
0619 16:57:43.366030 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:43.366033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:43.366088 (+ 55us) server_negotiation.cc:407] Connection header received
0619 16:57:43.366169 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:43.366170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:43.366176 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:43.366244 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:43.366420 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.366869 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.367797 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.368181 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.368263 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.368271 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:43.368426 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 16:57:43.368462 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 16:57:43.368536 16827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.365880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:43395 (local address 127.30.148.129:38443)
0619 16:57:43.366032 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:43.366041 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:43.366103 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:43.366258 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:43.366259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:43.366261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:43.366331 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.366333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.366957 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.366958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.367698 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:43.367699 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.368273 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.368274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:43.368343 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.368345 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:57:43.368413 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 16:57:43.368446 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":20,"thread_start_us":84,"threads_started":1}
I20260619 16:57:43.368853 16758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34d29c62b2e54f7fbcddc8c02be77a16" candidate_uuid: "37e5374952d345ae98007a6f7ca8c667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56add77b2187447394fb3e3e61c3d066" is_pre_election: true
I20260619 16:57:43.368999 16758 raft_consensus.cc:2454] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37e5374952d345ae98007a6f7ca8c667 in term 0.
I20260619 16:57:43.369127 16627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34d29c62b2e54f7fbcddc8c02be77a16" candidate_uuid: "37e5374952d345ae98007a6f7ca8c667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dbaab16898f492ebf5745538e34e704" is_pre_election: true
I20260619 16:57:43.369233 16432 leader_election.cc:302] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37e5374952d345ae98007a6f7ca8c667, 56add77b2187447394fb3e3e61c3d066; no voters:
I20260619 16:57:43.369278 16627 raft_consensus.cc:2454] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37e5374952d345ae98007a6f7ca8c667 in term 0.
I20260619 16:57:43.369364 16826 raft_consensus.cc:2781] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:57:43.369464 16826 raft_consensus.cc:482] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:57:43.369489 16826 raft_consensus.cc:3037] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:43.370148 16826 raft_consensus.cc:504] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dbaab16898f492ebf5745538e34e704" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 43395 } } peers { permanent_uuid: "37e5374952d345ae98007a6f7ca8c667" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36135 } } peers { permanent_uuid: "56add77b2187447394fb3e3e61c3d066" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34523 } }
I20260619 16:57:43.370299 16826 leader_election.cc:288] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [CANDIDATE]: Term 1 election: Requested vote from peers 7dbaab16898f492ebf5745538e34e704 (127.30.148.130:43395), 56add77b2187447394fb3e3e61c3d066 (127.30.148.131:34523)
I20260619 16:57:43.370430 16627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34d29c62b2e54f7fbcddc8c02be77a16" candidate_uuid: "37e5374952d345ae98007a6f7ca8c667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dbaab16898f492ebf5745538e34e704"
I20260619 16:57:43.370438 16758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34d29c62b2e54f7fbcddc8c02be77a16" candidate_uuid: "37e5374952d345ae98007a6f7ca8c667" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56add77b2187447394fb3e3e61c3d066"
I20260619 16:57:43.370520 16758 raft_consensus.cc:3037] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:43.370522 16627 raft_consensus.cc:3037] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:43.371210 16758 raft_consensus.cc:2454] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37e5374952d345ae98007a6f7ca8c667 in term 1.
I20260619 16:57:43.371351 16627 raft_consensus.cc:2454] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37e5374952d345ae98007a6f7ca8c667 in term 1.
I20260619 16:57:43.371359 16432 leader_election.cc:302] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37e5374952d345ae98007a6f7ca8c667, 56add77b2187447394fb3e3e61c3d066; no voters:
I20260619 16:57:43.371471 16826 raft_consensus.cc:2781] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:43.371621 16826 raft_consensus.cc:686] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [term 1 LEADER]: Becoming Leader. State: Replica: 37e5374952d345ae98007a6f7ca8c667, State: Running, Role: LEADER
I20260619 16:57:43.371831 16826 consensus_queue.cc:237] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dbaab16898f492ebf5745538e34e704" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 43395 } } peers { permanent_uuid: "37e5374952d345ae98007a6f7ca8c667" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36135 } } peers { permanent_uuid: "56add77b2187447394fb3e3e61c3d066" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34523 } }
I20260619 16:57:43.372596 16354 catalog_manager.cc:5495] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 reported cstate change: term changed from 0 to 1, leader changed from <none> to 37e5374952d345ae98007a6f7ca8c667 (127.30.148.129). New cstate: current_term: 1 leader_uuid: "37e5374952d345ae98007a6f7ca8c667" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dbaab16898f492ebf5745538e34e704" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 43395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37e5374952d345ae98007a6f7ca8c667" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56add77b2187447394fb3e3e61c3d066" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34523 } health_report { overall_health: UNKNOWN } } }
I20260619 16:57:43.396318 16816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.394067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55130 (local address 127.30.148.129:36135)
0619 16:57:43.394129 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:43.394131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:43.394139 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:57:43.394170 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:43.394170 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:43.394195 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:43.394266 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:57:43.394390 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.394840 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.395564 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:43.395761 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:43.395839 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:43.395990 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:57:43.396090 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:57:43.396181 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:43.396206 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 16:57:43.396226 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 16:57:43.397995 16627 raft_consensus.cc:1261] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704 [term 1 FOLLOWER]: Refusing update from remote peer 37e5374952d345ae98007a6f7ca8c667: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:43.397994 16758 raft_consensus.cc:1261] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066 [term 1 FOLLOWER]: Refusing update from remote peer 37e5374952d345ae98007a6f7ca8c667: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:43.398289 16826 consensus_queue.cc:1035] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56add77b2187447394fb3e3e61c3d066" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 34523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:43.398432 16828 consensus_queue.cc:1035] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dbaab16898f492ebf5745538e34e704" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 43395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 16:57:43.399526 16543 tablet.cc:2369] T 34d29c62b2e54f7fbcddc8c02be77a16 P 37e5374952d345ae98007a6f7ca8c667: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:57:43.400012 16805 tablet.cc:2369] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:57:43.400816 16831 mvcc.cc:205] Tried to move back new op lower bound from 7298614326875336704 to 7298614326771621888. Current Snapshot: MvccSnapshot[applied={T|T < 7298614326875336704}]
I20260619 16:57:43.403270 16353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:43.403370 16353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:43.404076 16353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df3ff796c9f142d3a4dcdc00386cb912: Sending DeleteTablet for 3 replicas of tablet 34d29c62b2e54f7fbcddc8c02be77a16
I20260619 16:57:43.404384 16607 tablet_service.cc:1514] Processing DeleteTablet for tablet 34d29c62b2e54f7fbcddc8c02be77a16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:43 UTC) from {username='slave'} at 127.0.0.1:43780
I20260619 16:57:43.404412 16738 tablet_service.cc:1514] Processing DeleteTablet for tablet 34d29c62b2e54f7fbcddc8c02be77a16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:43 UTC) from {username='slave'} at 127.0.0.1:57228
I20260619 16:57:43.404608 16841 tablet_replica.cc:330] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704: stopping tablet replica
I20260619 16:57:43.404699 16841 raft_consensus.cc:2229] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:43.404800 16841 raft_consensus.cc:2258] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:43.404825 16476 tablet_service.cc:1514] Processing DeleteTablet for tablet 34d29c62b2e54f7fbcddc8c02be77a16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:43 UTC) from {username='slave'} at 127.0.0.1:55126
I20260619 16:57:43.405107 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 16414
I20260619 16:57:43.405268 16841 ts_tablet_manager.cc:1905] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:43.406495 16842 tablet_replica.cc:330] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066: stopping tablet replica
I20260619 16:57:43.406667 16842 raft_consensus.cc:2229] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:43.406813 16842 raft_consensus.cc:2258] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:43.407270 16842 ts_tablet_manager.cc:1905] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:43.407474 16841 ts_tablet_manager.cc:1918] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:43.407547 16841 log.cc:1196] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/34d29c62b2e54f7fbcddc8c02be77a16
I20260619 16:57:43.407797 16841 ts_tablet_manager.cc:1939] T 34d29c62b2e54f7fbcddc8c02be77a16 P 7dbaab16898f492ebf5745538e34e704: Deleting consensus metadata
I20260619 16:57:43.408203 16338 catalog_manager.cc:4841] TS 7dbaab16898f492ebf5745538e34e704 (127.30.148.130:43395): tablet 34d29c62b2e54f7fbcddc8c02be77a16 (table test-table [id=039902410fb74b29b8ff5a4c3ddad9e1]) successfully deleted
I20260619 16:57:43.408686 16842 ts_tablet_manager.cc:1918] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:43.408768 16842 log.cc:1196] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/34d29c62b2e54f7fbcddc8c02be77a16
I20260619 16:57:43.408977 16842 ts_tablet_manager.cc:1939] T 34d29c62b2e54f7fbcddc8c02be77a16 P 56add77b2187447394fb3e3e61c3d066: Deleting consensus metadata
I20260619 16:57:43.409343 16341 catalog_manager.cc:4841] TS 56add77b2187447394fb3e3e61c3d066 (127.30.148.131:34523): tablet 34d29c62b2e54f7fbcddc8c02be77a16 (table test-table [id=039902410fb74b29b8ff5a4c3ddad9e1]) successfully deleted
W20260619 16:57:43.410727 16341 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:36135 (error 108)
I20260619 16:57:43.410881 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 16545
I20260619 16:57:43.412541 16813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:43.412391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36135 (local address 127.0.0.1:55146)
0619 16:57:43.412454 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:43.412502 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:36135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 16:57:43.412669 16341 catalog_manager.cc:4584] TS 37e5374952d345ae98007a6f7ca8c667 (127.30.148.129:36135): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 34d29c62b2e54f7fbcddc8c02be77a16: Network error: Client connection negotiation failed: client connection to 127.30.148.129:36135: connect: Connection refused (error 111)
I20260619 16:57:43.415525 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 16676
I20260619 16:57:43.419875 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 16323
2026-06-19T16:57:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (822 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:57:43 dist-test-slave-xb9n krb5kdc[16847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:57:43 dist-test-slave-xb9n krb5kdc[16847](info): set up 2 sockets
Jun 19 16:57:43 dist-test-slave-xb9n krb5kdc[16847](info): commencing operation
krb5kdc: starting...
W20260619 16:57:44.452904 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:57:44 dist-test-slave-xb9n krb5kdc[16847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:57:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:57:44.562390 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:41843
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:41843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:44.639767 16860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:44.640044 16860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:44.640084 16860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:44.642208 16860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:44.642264 16860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:44.642287 16860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:44.642315 16860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:44.642345 16860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:44.644579 16860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:41843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:41843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:44.644835 16860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:44.645102 16860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:44.650025 16865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:44.650024 16866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:44.650156 16860 server_base.cc:1034] running on GCE node
W20260619 16:57:44.650110 16868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:44.650462 16860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:44.650775 16860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:44.651928 16860 hybrid_clock.cc:648] HybridClock initialized: now 1781888264651917 us; error 41 us; skew 500 ppm
Jun 19 16:57:44 dist-test-slave-xb9n krb5kdc[16847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888264, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:44.654220 16860 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:57:44.654615 16860 webserver.cc:460] Webserver started at http://127.30.148.190:33875/ using document root <none> and password file <none>
I20260619 16:57:44.654835 16860 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:44.654901 16860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:44.655046 16860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:44.656095 16860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "73e6c3af4535497b9b20f3ea20c66f40"
format_stamp: "Formatted at 2026-06-19 16:57:44 on dist-test-slave-xb9n"
server_key: "d190de1f028593075e764d6fdb6e284c"
server_key_iv: "dc4d929f64cd3f433c5081876a20b670"
server_key_version: "encryptionkey@0"
I20260619 16:57:44.656430 16860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "73e6c3af4535497b9b20f3ea20c66f40"
format_stamp: "Formatted at 2026-06-19 16:57:44 on dist-test-slave-xb9n"
server_key: "d190de1f028593075e764d6fdb6e284c"
server_key_iv: "dc4d929f64cd3f433c5081876a20b670"
server_key_version: "encryptionkey@0"
I20260619 16:57:44.657688 16860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:57:44.658347 16875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:44.658488 16860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:57:44.658603 16860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "73e6c3af4535497b9b20f3ea20c66f40"
format_stamp: "Formatted at 2026-06-19 16:57:44 on dist-test-slave-xb9n"
server_key: "d190de1f028593075e764d6fdb6e284c"
server_key_iv: "dc4d929f64cd3f433c5081876a20b670"
server_key_version: "encryptionkey@0"
I20260619 16:57:44.658671 16860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:44.670202 16860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:44.671797 16860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:44.671944 16860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:44.675806 16860 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:41843
I20260619 16:57:44.675836 16927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:41843 every 8 connection(s)
I20260619 16:57:44.676151 16860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:44.676334 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 16860
I20260619 16:57:44.676447 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:44.676651 31314 external_mini_cluster.cc:1428] Setting key fbbaf43528afb92d745c6745f1440266
I20260619 16:57:44.677181 16928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:44.680162 16928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40: Bootstrap starting.
I20260619 16:57:44.680732 16928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:44.681020 16928 log.cc:824] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:44.681740 16928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40: No bootstrap required, opened a new log
Jun 19 16:57:44 dist-test-slave-xb9n krb5kdc[16847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:44.683871 16928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e6c3af4535497b9b20f3ea20c66f40" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41843 } }
I20260619 16:57:44.684021 16928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:44.684046 16928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73e6c3af4535497b9b20f3ea20c66f40, State: Initialized, Role: FOLLOWER
I20260619 16:57:44.684177 16928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e6c3af4535497b9b20f3ea20c66f40" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41843 } }
I20260619 16:57:44.684258 16928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:44.684291 16928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:44.684342 16928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:44.684940 16928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e6c3af4535497b9b20f3ea20c66f40" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41843 } }
I20260619 16:57:44.685030 16928 leader_election.cc:302] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73e6c3af4535497b9b20f3ea20c66f40; no voters:
I20260619 16:57:44.685192 16928 leader_election.cc:288] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:44.685232 16933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:44.685439 16933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [term 1 LEADER]: Becoming Leader. State: Replica: 73e6c3af4535497b9b20f3ea20c66f40, State: Running, Role: LEADER
I20260619 16:57:44.685484 16928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:44.685523 16933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e6c3af4535497b9b20f3ea20c66f40" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41843 } }
I20260619 16:57:44.685829 16933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73e6c3af4535497b9b20f3ea20c66f40. Latest consensus state: current_term: 1 leader_uuid: "73e6c3af4535497b9b20f3ea20c66f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e6c3af4535497b9b20f3ea20c66f40" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41843 } } }
I20260619 16:57:44.685901 16933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:44.686133 16934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73e6c3af4535497b9b20f3ea20c66f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73e6c3af4535497b9b20f3ea20c66f40" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 41843 } } }
I20260619 16:57:44.686219 16934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40 [sys.catalog]: This master's current role is: LEADER
W20260619 16:57:44.687476 16948 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:57:44.687543 16948 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:57:44.687588 16949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:44.687624 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:44.677105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.30.148.190:41843)
0619 16:57:44.677292 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:44.677301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:44.677331 (+ 30us) server_negotiation.cc:407] Connection header received
0619 16:57:44.677870 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:44.677878 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:44.677944 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:44.678087 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:44.678640 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:44.679964 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:44.680699 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:44.681282 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:44.681396 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:44.683920 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:44.683923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:44.683923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:44.683932 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:44.685914 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:44.686312 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:44.686315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:44.686315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:44.686415 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:44.686631 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:44.686632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:44.686632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:44.686941 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:44.687125 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:44.687247 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 16:57:44.687374 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260619 16:57:44.687747 16949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:44.689162 16949 catalog_manager.cc:1347] Generated new cluster ID: 67b1768bc1e54a478abfd33d39cb063a
I20260619 16:57:44.689215 16949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:44.712033 16949 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:44.712725 16949 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:44.724190 16949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40: Generated new TSK 0
I20260619 16:57:44.724361 16949 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:57:44.749716 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41843
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:44.827899 16954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:44.828135 16954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:44.828176 16954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:44.829286 16954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:44.829346 16954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:44.829416 16954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:57:44.830797 16954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:44.831030 16954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:44.831259 16954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:44.835839 16962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:44.835856 16959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:44.835918 16960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:44.835902 16954 server_base.cc:1034] running on GCE node
I20260619 16:57:44.836194 16954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:44.836442 16954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:44.837598 16954 hybrid_clock.cc:648] HybridClock initialized: now 1781888264837579 us; error 40 us; skew 500 ppm
Jun 19 16:57:44 dist-test-slave-xb9n krb5kdc[16847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888264, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:44.839927 16954 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:57:44.840310 16954 webserver.cc:460] Webserver started at http://127.30.148.129:43569/ using document root <none> and password file <none>
I20260619 16:57:44.840525 16954 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:44.840605 16954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:44.840742 16954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:44.841754 16954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "45dbbfe191fb44e8890c7a81fedcb1c5"
format_stamp: "Formatted at 2026-06-19 16:57:44 on dist-test-slave-xb9n"
server_key: "db3c033bbe64e0afc9c2e610284fb554"
server_key_iv: "6846d52ac712f8d1b9655f086c5a37c7"
server_key_version: "encryptionkey@0"
I20260619 16:57:44.842087 16954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "45dbbfe191fb44e8890c7a81fedcb1c5"
format_stamp: "Formatted at 2026-06-19 16:57:44 on dist-test-slave-xb9n"
server_key: "db3c033bbe64e0afc9c2e610284fb554"
server_key_iv: "6846d52ac712f8d1b9655f086c5a37c7"
server_key_version: "encryptionkey@0"
I20260619 16:57:44.843307 16954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:57:44.843961 16969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:44.844125 16954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:44.844213 16954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "45dbbfe191fb44e8890c7a81fedcb1c5"
format_stamp: "Formatted at 2026-06-19 16:57:44 on dist-test-slave-xb9n"
server_key: "db3c033bbe64e0afc9c2e610284fb554"
server_key_iv: "6846d52ac712f8d1b9655f086c5a37c7"
server_key_version: "encryptionkey@0"
I20260619 16:57:44.844285 16954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:44.861873 16954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:44.863722 16954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:44.863868 16954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:44.864068 16954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:44.864347 16954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:44.864398 16954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:44.864439 16954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:44.864473 16954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:44.869997 16954 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:36657
I20260619 16:57:44.870045 17082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:36657 every 8 connection(s)
I20260619 16:57:44.870323 16954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:57:44.874150 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 16954
I20260619 16:57:44.874249 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:57:44.874478 31314 external_mini_cluster.cc:1428] Setting key f1162911944eca85e3e8cc3a02659f7e
Jun 19 16:57:44 dist-test-slave-xb9n krb5kdc[16847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888264, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:44.879966 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:44.870949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:44449 (local address 127.30.148.190:41843)
0619 16:57:44.871020 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:44.871022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:44.871201 (+ 179us) server_negotiation.cc:407] Connection header received
0619 16:57:44.871636 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:44.871637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:44.871647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:44.871697 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:44.872526 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:44.873045 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:44.874008 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:44.874355 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:44.874435 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:44.876946 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:44.876948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:44.876948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:44.876955 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:44.878554 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:44.878957 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:44.878958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:44.878959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:44.879045 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:44.879325 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:44.879326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:44.879326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:44.879467 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:44.879535 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:44.879770 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 16:57:44.879853 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 16:57:44.879982 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:44.870925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41843 (local address 127.30.148.129:44449)
0619 16:57:44.871089 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:44.871125 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:44.871560 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:44.871746 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:44.871748 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:44.872276 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:44.872450 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:44.872455 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:44.873143 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:44.873144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:44.873921 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:44.873923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:44.874440 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:44.874440 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:44.874560 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:44.875074 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:44.875084 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:44.876840 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:44.878646 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:44.878652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:44.878653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:44.878867 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:44.879140 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:44.879141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:44.879141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:44.879248 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:44.879537 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:44.879542 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:44.879650 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 16:57:44.879727 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 16:57:44.879837 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260619 16:57:44.880504 17083 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41843
I20260619 16:57:44.880617 17083 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:44.880782 17083 heartbeater.cc:511] Master 127.30.148.190:41843 requested a full tablet report, sending...
I20260619 16:57:44.881294 16892 ts_manager.cc:194] Registered new tserver with Master: 45dbbfe191fb44e8890c7a81fedcb1c5 (127.30.148.129:36657)
I20260619 16:57:44.881893 16892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:44449
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:57:44.899374 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41843
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:44.976725 17088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:44.976953 17088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:44.976990 17088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:44.978111 17088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:44.978173 17088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:44.978217 17088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:57:44.979630 17088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:44.979873 17088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:44.980082 17088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:44.984983 17096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:44.985060 17088 server_base.cc:1034] running on GCE node
W20260619 16:57:44.984998 17094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:44.985188 17093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:44.985440 17088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:44.985675 17088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:44.986842 17088 hybrid_clock.cc:648] HybridClock initialized: now 1781888264986811 us; error 49 us; skew 500 ppm
Jun 19 16:57:44 dist-test-slave-xb9n krb5kdc[16847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888264, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:44.989265 17088 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:57:44.989738 17088 webserver.cc:460] Webserver started at http://127.30.148.130:40805/ using document root <none> and password file <none>
I20260619 16:57:44.989959 17088 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:44.990021 17088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:44.990151 17088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:44.991127 17088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "3fd93892b22a427c9687b8df4111ae9a"
format_stamp: "Formatted at 2026-06-19 16:57:44 on dist-test-slave-xb9n"
server_key: "54199a2329d9ea93cb2a2e42c681ae70"
server_key_iv: "21476c28f3c1b1df8de19b7b86140451"
server_key_version: "encryptionkey@0"
I20260619 16:57:44.991470 17088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "3fd93892b22a427c9687b8df4111ae9a"
format_stamp: "Formatted at 2026-06-19 16:57:44 on dist-test-slave-xb9n"
server_key: "54199a2329d9ea93cb2a2e42c681ae70"
server_key_iv: "21476c28f3c1b1df8de19b7b86140451"
server_key_version: "encryptionkey@0"
I20260619 16:57:44.992746 17088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:57:44.993525 17103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:44.993681 17088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:57:44.993764 17088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "3fd93892b22a427c9687b8df4111ae9a"
format_stamp: "Formatted at 2026-06-19 16:57:44 on dist-test-slave-xb9n"
server_key: "54199a2329d9ea93cb2a2e42c681ae70"
server_key_iv: "21476c28f3c1b1df8de19b7b86140451"
server_key_version: "encryptionkey@0"
I20260619 16:57:44.993842 17088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:45.009012 17088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:45.010905 17088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:45.011068 17088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:45.011296 17088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:45.011590 17088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:45.011646 17088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.011682 17088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:45.011708 17088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.017379 17088 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:34349
I20260619 16:57:45.017428 17216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:34349 every 8 connection(s)
I20260619 16:57:45.017712 17088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:57:45 dist-test-slave-xb9n krb5kdc[16847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888264, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:45.023661 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 17088
I20260619 16:57:45.023772 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:57:45.023979 31314 external_mini_cluster.cc:1428] Setting key 7e33b00903f3c0b9e1000468ecab845a
I20260619 16:57:45.027155 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.018349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41843 (local address 127.30.148.130:34907)
0619 16:57:45.018503 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.018541 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.019016 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.019231 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.019235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.019782 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:45.019954 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.019958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.020589 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.020590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.021330 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.021332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.021740 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.021741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.021839 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.022316 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:45.022324 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:45.023805 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:45.025664 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:45.025671 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:45.025673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:45.025988 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:45.026215 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:45.026216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:45.026216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:45.026309 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:45.026689 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:45.026699 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:45.026818 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.026896 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.027002 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260619 16:57:45.027174 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.018398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:34907 (local address 127.30.148.190:41843)
0619 16:57:45.018475 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.018477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.018650 (+ 173us) server_negotiation.cc:407] Connection header received
0619 16:57:45.019093 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.019093 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.019102 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.019162 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:45.020043 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.020512 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.021468 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.021674 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.021726 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.023889 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:45.023891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:45.023891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:45.023897 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:45.025585 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:45.026065 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:45.026066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:45.026067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:45.026146 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:45.026368 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:45.026369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:45.026370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:45.026608 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:45.026679 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.026937 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.027030 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 16:57:45.027741 17217 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41843
I20260619 16:57:45.027820 17217 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:45.028002 17217 heartbeater.cc:511] Master 127.30.148.190:41843 requested a full tablet report, sending...
I20260619 16:57:45.028430 16892 ts_manager.cc:194] Registered new tserver with Master: 3fd93892b22a427c9687b8df4111ae9a (127.30.148.130:34349)
I20260619 16:57:45.028805 16892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:34907
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:57:45.047917 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41843
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:45.126965 17222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:45.127180 17222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:45.127221 17222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:45.128351 17222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:45.128413 17222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:45.128458 17222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:57:45.129889 17222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:45.130138 17222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:45.130364 17222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:45.135152 17228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:45.135161 17227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:45.135249 17222 server_base.cc:1034] running on GCE node
W20260619 16:57:45.135227 17230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:45.135633 17222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:45.135872 17222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:45.137024 17222 hybrid_clock.cc:648] HybridClock initialized: now 1781888265137017 us; error 33 us; skew 500 ppm
Jun 19 16:57:45 dist-test-slave-xb9n krb5kdc[16847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888265, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:45.139410 17222 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:57:45.139819 17222 webserver.cc:460] Webserver started at http://127.30.148.131:37331/ using document root <none> and password file <none>
I20260619 16:57:45.140038 17222 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:45.140100 17222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:45.140240 17222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:45.141193 17222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "aa6d7135295047dda1a5376bc0ce600f"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "8a30886d0a737125d9663771908d57e6"
server_key_iv: "a1870e8a4bb5c88352cb185d47430bcd"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.141548 17222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "aa6d7135295047dda1a5376bc0ce600f"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "8a30886d0a737125d9663771908d57e6"
server_key_iv: "a1870e8a4bb5c88352cb185d47430bcd"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.142802 17222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:57:45.143488 17237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.143640 17222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:57:45.143726 17222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "aa6d7135295047dda1a5376bc0ce600f"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "8a30886d0a737125d9663771908d57e6"
server_key_iv: "a1870e8a4bb5c88352cb185d47430bcd"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.143802 17222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:45.152040 17222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:45.153801 17222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:45.153939 17222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:45.154136 17222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:45.154404 17222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:45.154453 17222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.154492 17222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:45.154525 17222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.159924 17222 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:41617
I20260619 16:57:45.159961 17350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:41617 every 8 connection(s)
I20260619 16:57:45.160209 17222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:57:45.162026 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 17222
I20260619 16:57:45.162134 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:57:45.162359 31314 external_mini_cluster.cc:1428] Setting key a01aa24720595b0ff34c1d5bbaa77dcc
Jun 19 16:57:45 dist-test-slave-xb9n krb5kdc[16847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888265, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:45.168696 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.160764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:34407 (local address 127.30.148.190:41843)
0619 16:57:45.160804 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.160806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.160974 (+ 168us) server_negotiation.cc:407] Connection header received
0619 16:57:45.161488 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.161489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.161498 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.161551 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:45.162529 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.163260 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.164225 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.164407 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.164478 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.166509 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:45.166510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:45.166510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:45.166532 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:45.167669 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:45.167994 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:45.167995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:45.167995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:45.168040 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:45.168246 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:45.168246 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:45.168247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:45.168335 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:45.168382 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.168582 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.168636 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:57:45.168888 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.160713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:41843 (local address 127.30.148.131:34407)
0619 16:57:45.160856 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.160892 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.161350 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.161619 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.161622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.162193 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:45.162422 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.162428 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.163335 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.163336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.164148 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.164150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.164507 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.164507 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.164600 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.165074 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:45.165088 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:45.166441 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:45.167733 (+ 1292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:45.167738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:45.167740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:45.167935 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:45.168102 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:45.168102 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:45.168102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:45.168197 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:45.168383 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:45.168387 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:45.168484 (+ 97us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.168559 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.168662 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260619 16:57:45.169312 17351 heartbeater.cc:347] Connected to a master server at 127.30.148.190:41843
I20260619 16:57:45.169420 17351 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:45.169585 17351 heartbeater.cc:511] Master 127.30.148.190:41843 requested a full tablet report, sending...
I20260619 16:57:45.169919 16892 ts_manager.cc:194] Registered new tserver with Master: aa6d7135295047dda1a5376bc0ce600f (127.30.148.131:41617)
I20260619 16:57:45.170266 16892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:34407
I20260619 16:57:45.173893 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:57:45.180294 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.175111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34158 (local address 127.30.148.190:41843)
0619 16:57:45.175174 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.175176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.175274 (+ 98us) server_negotiation.cc:407] Connection header received
0619 16:57:45.175317 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.175317 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.175325 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.175367 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:45.175839 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.176268 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.176927 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.177090 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.177133 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.178257 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:45.178258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:45.178258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:45.178263 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:45.179364 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:45.179703 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:45.179704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:45.179704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:45.179749 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:45.179885 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:45.179885 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:45.179886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:45.179987 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:45.180063 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.180152 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.180195 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 16:57:45.181726 16892 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:57:45.183128 16892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:57:45.189136 17359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.184869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:41617 (local address 127.0.0.1:55850)
0619 16:57:45.185109 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.185146 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.185216 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.185515 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.185518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.185520 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:45.185589 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.185590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.186823 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.186824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.188080 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.188083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.188835 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.188836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.188914 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.188926 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.189017 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.189069 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":57,"threads_started":1}
I20260619 16:57:45.189144 17361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.184952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55850 (local address 127.30.148.131:41617)
0619 16:57:45.185100 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.185105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.185312 (+ 207us) server_negotiation.cc:407] Connection header received
0619 16:57:45.185400 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.185403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.185418 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.185496 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:45.185670 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.186726 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.188162 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.188740 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.188832 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.188886 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.189033 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.189068 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260619 16:57:45.189560 17360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.184906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:34349 (local address 127.0.0.1:33920)
0619 16:57:45.185174 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.185196 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.185259 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.185669 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.185670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.185673 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:45.185715 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.185716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.187330 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.187331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.188366 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.188368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.189291 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.189292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.189378 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.189384 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.189443 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.189483 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":9,"thread_start_us":57,"threads_started":1}
I20260619 16:57:45.189589 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.185020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33920 (local address 127.30.148.130:34349)
0619 16:57:45.185448 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.185453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.185461 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:57:45.185497 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.185499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.185514 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.185580 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:45.185844 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.187048 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.188464 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.189134 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.189206 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.189254 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.189460 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.189502 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":82,"threads_started":1}
I20260619 16:57:45.189749 17362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.185058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36657 (local address 127.0.0.1:52310)
0619 16:57:45.185229 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.185238 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.185291 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.185764 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.185765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.185767 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:45.185833 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.185835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.187222 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.187223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.188349 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.188352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.189295 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.189296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.189378 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.189386 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.189438 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.189483 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":3,"thread_start_us":40,"threads_started":1}
I20260619 16:57:45.189891 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.185287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52310 (local address 127.30.148.129:36657)
0619 16:57:45.185524 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.185529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.185538 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:57:45.185586 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.185588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.185604 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.185686 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:45.185915 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.187125 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.188431 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.189134 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.189206 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.189254 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.189525 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.189565 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":99,"threads_started":1}
I20260619 16:57:45.190142 17285 tablet_service.cc:1467] Processing CreateTablet for tablet 77be3b1ae4ce4ebbad9a09ec0387c757 (DEFAULT_TABLE table=test-table [id=730cf7e17ba14b6b99a88d494311a833]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:45.190222 17151 tablet_service.cc:1467] Processing CreateTablet for tablet 77be3b1ae4ce4ebbad9a09ec0387c757 (DEFAULT_TABLE table=test-table [id=730cf7e17ba14b6b99a88d494311a833]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:45.190476 17285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77be3b1ae4ce4ebbad9a09ec0387c757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:45.190407 17017 tablet_service.cc:1467] Processing CreateTablet for tablet 77be3b1ae4ce4ebbad9a09ec0387c757 (DEFAULT_TABLE table=test-table [id=730cf7e17ba14b6b99a88d494311a833]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:45.190529 17151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77be3b1ae4ce4ebbad9a09ec0387c757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:45.190666 17017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77be3b1ae4ce4ebbad9a09ec0387c757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:45.192914 17365 tablet_bootstrap.cc:492] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f: Bootstrap starting.
I20260619 16:57:45.193656 17366 tablet_bootstrap.cc:492] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5: Bootstrap starting.
I20260619 16:57:45.193760 17365 tablet_bootstrap.cc:654] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:45.193830 17367 tablet_bootstrap.cc:492] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a: Bootstrap starting.
I20260619 16:57:45.194331 17367 tablet_bootstrap.cc:654] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:45.194335 17365 log.cc:824] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:45.194417 17366 tablet_bootstrap.cc:654] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:45.194866 17367 log.cc:824] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:45.194929 17366 log.cc:824] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:45.195112 17365 tablet_bootstrap.cc:492] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f: No bootstrap required, opened a new log
I20260619 16:57:45.195190 17365 ts_tablet_manager.cc:1397] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 16:57:45.195639 17367 tablet_bootstrap.cc:492] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a: No bootstrap required, opened a new log
I20260619 16:57:45.195719 17367 ts_tablet_manager.cc:1397] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:57:45.197126 17365 raft_consensus.cc:348] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6d7135295047dda1a5376bc0ce600f" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41617 } } peers { permanent_uuid: "3fd93892b22a427c9687b8df4111ae9a" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34349 } } peers { permanent_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36657 } }
I20260619 16:57:45.197316 17365 raft_consensus.cc:374] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:45.197386 17365 raft_consensus.cc:729] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa6d7135295047dda1a5376bc0ce600f, State: Initialized, Role: FOLLOWER
I20260619 16:57:45.197513 17365 consensus_queue.cc:260] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6d7135295047dda1a5376bc0ce600f" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41617 } } peers { permanent_uuid: "3fd93892b22a427c9687b8df4111ae9a" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34349 } } peers { permanent_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36657 } }
I20260619 16:57:45.197764 17365 ts_tablet_manager.cc:1428] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 16:57:45.197786 17351 heartbeater.cc:503] Master 127.30.148.190:41843 was elected leader, sending a full tablet report...
I20260619 16:57:45.197714 17367 raft_consensus.cc:348] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6d7135295047dda1a5376bc0ce600f" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41617 } } peers { permanent_uuid: "3fd93892b22a427c9687b8df4111ae9a" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34349 } } peers { permanent_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36657 } }
I20260619 16:57:45.197880 17367 raft_consensus.cc:374] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:45.197921 17367 raft_consensus.cc:729] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fd93892b22a427c9687b8df4111ae9a, State: Initialized, Role: FOLLOWER
I20260619 16:57:45.198024 17367 consensus_queue.cc:260] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6d7135295047dda1a5376bc0ce600f" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41617 } } peers { permanent_uuid: "3fd93892b22a427c9687b8df4111ae9a" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34349 } } peers { permanent_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36657 } }
I20260619 16:57:45.198252 17367 ts_tablet_manager.cc:1428] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 16:57:45.198382 17217 heartbeater.cc:503] Master 127.30.148.190:41843 was elected leader, sending a full tablet report...
I20260619 16:57:45.198684 17366 tablet_bootstrap.cc:492] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5: No bootstrap required, opened a new log
I20260619 16:57:45.198751 17366 ts_tablet_manager.cc:1397] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 16:57:45.200115 17366 raft_consensus.cc:348] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6d7135295047dda1a5376bc0ce600f" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41617 } } peers { permanent_uuid: "3fd93892b22a427c9687b8df4111ae9a" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34349 } } peers { permanent_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36657 } }
I20260619 16:57:45.200240 17366 raft_consensus.cc:374] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:45.200263 17366 raft_consensus.cc:729] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45dbbfe191fb44e8890c7a81fedcb1c5, State: Initialized, Role: FOLLOWER
I20260619 16:57:45.200380 17366 consensus_queue.cc:260] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6d7135295047dda1a5376bc0ce600f" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41617 } } peers { permanent_uuid: "3fd93892b22a427c9687b8df4111ae9a" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34349 } } peers { permanent_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36657 } }
I20260619 16:57:45.200614 17366 ts_tablet_manager.cc:1428] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:57:45.200661 17083 heartbeater.cc:503] Master 127.30.148.190:41843 was elected leader, sending a full tablet report...
W20260619 16:57:45.268270 17218 tablet.cc:2369] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:57:45.306552 17372 raft_consensus.cc:482] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:57:45.306675 17372 raft_consensus.cc:504] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6d7135295047dda1a5376bc0ce600f" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41617 } } peers { permanent_uuid: "3fd93892b22a427c9687b8df4111ae9a" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34349 } } peers { permanent_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36657 } }
I20260619 16:57:45.306974 17372 leader_election.cc:288] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa6d7135295047dda1a5376bc0ce600f (127.30.148.131:41617), 45dbbfe191fb44e8890c7a81fedcb1c5 (127.30.148.129:36657)
I20260619 16:57:45.309842 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.307217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:58493 (local address 127.30.148.129:36657)
0619 16:57:45.307261 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.307264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.307426 (+ 162us) server_negotiation.cc:407] Connection header received
0619 16:57:45.307468 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.307469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.307479 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.307537 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:45.307704 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.308170 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.309098 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.309493 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.309595 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.309603 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.309729 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.309776 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:57:45.309852 17361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.307221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:35075 (local address 127.30.148.131:41617)
0619 16:57:45.307261 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.307264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.307273 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:57:45.307303 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.307304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.307313 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.307389 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:45.307554 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.308099 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.309070 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.309495 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.309565 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.309575 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.309751 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.309787 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:57:45.309875 17374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.307145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36657 (local address 127.30.148.130:58493)
0619 16:57:45.307349 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.307360 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.307418 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.307554 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.307555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.307557 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:45.307638 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.307640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.308248 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.308248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.308988 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.308990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.309599 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.309600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.309661 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.309664 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.309706 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.309788 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":12,"thread_start_us":99,"threads_started":1}
I20260619 16:57:45.309880 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.307144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:41617 (local address 127.30.148.130:35075)
0619 16:57:45.307195 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.307207 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.307249 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.307394 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.307395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.307408 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:45.307481 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.307482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.308174 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.308175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.308979 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.308981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.309586 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.309587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.309662 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.309665 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.309735 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.309804 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":77}
I20260619 16:57:45.310303 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77be3b1ae4ce4ebbad9a09ec0387c757" candidate_uuid: "3fd93892b22a427c9687b8df4111ae9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5" is_pre_election: true
I20260619 16:57:45.310457 17037 raft_consensus.cc:2454] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fd93892b22a427c9687b8df4111ae9a in term 0.
I20260619 16:57:45.310408 17305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77be3b1ae4ce4ebbad9a09ec0387c757" candidate_uuid: "3fd93892b22a427c9687b8df4111ae9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa6d7135295047dda1a5376bc0ce600f" is_pre_election: true
I20260619 16:57:45.310561 17305 raft_consensus.cc:2454] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fd93892b22a427c9687b8df4111ae9a in term 0.
I20260619 16:57:45.310674 17107 leader_election.cc:302] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fd93892b22a427c9687b8df4111ae9a, 45dbbfe191fb44e8890c7a81fedcb1c5; no voters:
I20260619 16:57:45.310833 17372 raft_consensus.cc:2781] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:57:45.310871 17372 raft_consensus.cc:482] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:57:45.310886 17372 raft_consensus.cc:3037] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:45.311533 17372 raft_consensus.cc:504] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6d7135295047dda1a5376bc0ce600f" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41617 } } peers { permanent_uuid: "3fd93892b22a427c9687b8df4111ae9a" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34349 } } peers { permanent_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36657 } }
I20260619 16:57:45.311722 17372 leader_election.cc:288] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [CANDIDATE]: Term 1 election: Requested vote from peers aa6d7135295047dda1a5376bc0ce600f (127.30.148.131:41617), 45dbbfe191fb44e8890c7a81fedcb1c5 (127.30.148.129:36657)
I20260619 16:57:45.311836 17037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77be3b1ae4ce4ebbad9a09ec0387c757" candidate_uuid: "3fd93892b22a427c9687b8df4111ae9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5"
I20260619 16:57:45.311868 17305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77be3b1ae4ce4ebbad9a09ec0387c757" candidate_uuid: "3fd93892b22a427c9687b8df4111ae9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa6d7135295047dda1a5376bc0ce600f"
I20260619 16:57:45.311910 17037 raft_consensus.cc:3037] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:45.311975 17305 raft_consensus.cc:3037] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:45.312594 17037 raft_consensus.cc:2454] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fd93892b22a427c9687b8df4111ae9a in term 1.
I20260619 16:57:45.312743 17305 raft_consensus.cc:2454] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fd93892b22a427c9687b8df4111ae9a in term 1.
I20260619 16:57:45.312754 17107 leader_election.cc:302] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fd93892b22a427c9687b8df4111ae9a, 45dbbfe191fb44e8890c7a81fedcb1c5; no voters:
I20260619 16:57:45.312841 17372 raft_consensus.cc:2781] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:45.312987 17372 raft_consensus.cc:686] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 1 LEADER]: Becoming Leader. State: Replica: 3fd93892b22a427c9687b8df4111ae9a, State: Running, Role: LEADER
I20260619 16:57:45.313078 17372 consensus_queue.cc:237] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6d7135295047dda1a5376bc0ce600f" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41617 } } peers { permanent_uuid: "3fd93892b22a427c9687b8df4111ae9a" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34349 } } peers { permanent_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36657 } }
I20260619 16:57:45.313892 16892 catalog_manager.cc:5495] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fd93892b22a427c9687b8df4111ae9a (127.30.148.130). New cstate: current_term: 1 leader_uuid: "3fd93892b22a427c9687b8df4111ae9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa6d7135295047dda1a5376bc0ce600f" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fd93892b22a427c9687b8df4111ae9a" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 34349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36657 } health_report { overall_health: UNKNOWN } } }
I20260619 16:57:45.333881 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.331811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.30.148.130:34349)
0619 16:57:45.331850 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.331852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.331864 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:57:45.331933 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.331934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.331948 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.331997 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:57:45.332125 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.332547 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.333224 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.333440 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.333490 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.333624 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:57:45.333698 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:57:45.333763 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.333811 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.333819 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:57:45.335528 17037 raft_consensus.cc:1261] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 45dbbfe191fb44e8890c7a81fedcb1c5 [term 1 FOLLOWER]: Refusing update from remote peer 3fd93892b22a427c9687b8df4111ae9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:45.335552 17305 raft_consensus.cc:1261] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f [term 1 FOLLOWER]: Refusing update from remote peer 3fd93892b22a427c9687b8df4111ae9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:45.335918 17375 consensus_queue.cc:1035] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "45dbbfe191fb44e8890c7a81fedcb1c5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:45.336041 17372 consensus_queue.cc:1035] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa6d7135295047dda1a5376bc0ce600f" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 41617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:45.338619 17379 mvcc.cc:205] Tried to move back new op lower bound from 7298614334811693056 to 7298614334722875392. Current Snapshot: MvccSnapshot[applied={T|T < 7298614334811693056}]
I20260619 16:57:45.338706 17380 mvcc.cc:205] Tried to move back new op lower bound from 7298614334811693056 to 7298614334722875392. Current Snapshot: MvccSnapshot[applied={T|T < 7298614334811693056}]
I20260619 16:57:45.340907 16891 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:45.341001 16891 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:45.341704 16891 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 73e6c3af4535497b9b20f3ea20c66f40: Sending DeleteTablet for 3 replicas of tablet 77be3b1ae4ce4ebbad9a09ec0387c757
I20260619 16:57:45.341989 17285 tablet_service.cc:1514] Processing DeleteTablet for tablet 77be3b1ae4ce4ebbad9a09ec0387c757 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:45 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:55850
I20260619 16:57:45.342022 17017 tablet_service.cc:1514] Processing DeleteTablet for tablet 77be3b1ae4ce4ebbad9a09ec0387c757 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:45 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:52310
I20260619 16:57:45.342340 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 16954
I20260619 16:57:45.345055 17151 tablet_service.cc:1514] Processing DeleteTablet for tablet 77be3b1ae4ce4ebbad9a09ec0387c757 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:45 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:33920
I20260619 16:57:45.345239 17390 tablet_replica.cc:330] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a: stopping tablet replica
I20260619 16:57:45.345335 17390 raft_consensus.cc:2229] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:57:45.345472 17390 raft_consensus.cc:2258] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:45.345842 17390 ts_tablet_manager.cc:1905] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:45.347812 17390 ts_tablet_manager.cc:1918] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:45.347887 17390 log.cc:1196] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/77be3b1ae4ce4ebbad9a09ec0387c757
I20260619 16:57:45.348158 17390 ts_tablet_manager.cc:1939] T 77be3b1ae4ce4ebbad9a09ec0387c757 P 3fd93892b22a427c9687b8df4111ae9a: Deleting consensus metadata
I20260619 16:57:45.348620 16879 catalog_manager.cc:4841] TS 3fd93892b22a427c9687b8df4111ae9a (127.30.148.130:34349): tablet 77be3b1ae4ce4ebbad9a09ec0387c757 (table test-table [id=730cf7e17ba14b6b99a88d494311a833]) successfully deleted
W20260619 16:57:45.349010 16879 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:36657 (error 108)
I20260619 16:57:45.349216 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 17088
I20260619 16:57:45.349776 17362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.349384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36657 (local address 127.0.0.1:52320)
0619 16:57:45.349710 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.349745 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:36657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307}
W20260619 16:57:45.349886 16879 catalog_manager.cc:4584] TS 45dbbfe191fb44e8890c7a81fedcb1c5 (127.30.148.129:36657): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77be3b1ae4ce4ebbad9a09ec0387c757: Network error: Client connection negotiation failed: client connection to 127.30.148.129:36657: connect: Connection refused (error 111)
I20260619 16:57:45.350229 17388 tablet_replica.cc:330] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f: stopping tablet replica
I20260619 16:57:45.350337 17388 raft_consensus.cc:2229] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:45.350440 17388 raft_consensus.cc:2258] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:45.350808 17388 ts_tablet_manager.cc:1905] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:45.352682 17388 ts_tablet_manager.cc:1918] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:45.352756 17388 log.cc:1196] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/77be3b1ae4ce4ebbad9a09ec0387c757
I20260619 16:57:45.353016 17388 ts_tablet_manager.cc:1939] T 77be3b1ae4ce4ebbad9a09ec0387c757 P aa6d7135295047dda1a5376bc0ce600f: Deleting consensus metadata
I20260619 16:57:45.353400 16876 catalog_manager.cc:4841] TS aa6d7135295047dda1a5376bc0ce600f (127.30.148.131:41617): tablet 77be3b1ae4ce4ebbad9a09ec0387c757 (table test-table [id=730cf7e17ba14b6b99a88d494311a833]) successfully deleted
I20260619 16:57:45.355294 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 17222
I20260619 16:57:45.360090 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 16860
2026-06-19T16:57:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1940 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T16:57:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:45Z Disabled control of system clock
I20260619 16:57:45.386377 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33789
--webserver_interface=127.30.148.190
--webserver_port=0
--builtin_ntp_servers=127.30.148.148:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:33789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 16:57:45.463848 17395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:45.464033 17395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:45.464052 17395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:45.465164 17395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:45.465204 17395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:45.465216 17395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:45.465229 17395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:45.465273 17395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:45.466595 17395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:33789
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:45.466761 17395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:45.467052 17395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:45.471637 17400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:45.471666 17395 server_base.cc:1034] running on GCE node
W20260619 16:57:45.471601 17403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:45.471638 17401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:45.472038 17395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:45.472296 17395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:45.473415 17395 hybrid_clock.cc:648] HybridClock initialized: now 1781888265473399 us; error 28 us; skew 500 ppm
I20260619 16:57:45.474538 17395 webserver.cc:460] Webserver started at http://127.30.148.190:36579/ using document root <none> and password file <none>
I20260619 16:57:45.474726 17395 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:45.474766 17395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:45.474848 17395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:45.475746 17395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "931c15f4758f4b0d96451849bcd45e03"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "0ae4d9c21d76596e18746dad5de95033"
server_key_iv: "5189e3f7132534a1b020158fb84be001"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.476020 17395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "931c15f4758f4b0d96451849bcd45e03"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "0ae4d9c21d76596e18746dad5de95033"
server_key_iv: "5189e3f7132534a1b020158fb84be001"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.477169 17395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 16:57:45.477897 17409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.478047 17395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.478150 17395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "931c15f4758f4b0d96451849bcd45e03"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "0ae4d9c21d76596e18746dad5de95033"
server_key_iv: "5189e3f7132534a1b020158fb84be001"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.478204 17395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:45.506068 17395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:45.506299 17395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:45.506397 17395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:45.510072 17395 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:33789
I20260619 16:57:45.510100 17461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:33789 every 8 connection(s)
I20260619 16:57:45.510430 17395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:45.511329 17462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:45.513347 17462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03: Bootstrap starting.
I20260619 16:57:45.513962 17462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:45.514253 17462 log.cc:824] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:45.514868 17462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03: No bootstrap required, opened a new log
I20260619 16:57:45.516352 17462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "931c15f4758f4b0d96451849bcd45e03" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33789 } }
I20260619 16:57:45.516474 17462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:45.516495 17462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931c15f4758f4b0d96451849bcd45e03, State: Initialized, Role: FOLLOWER
I20260619 16:57:45.516655 17462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "931c15f4758f4b0d96451849bcd45e03" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33789 } }
I20260619 16:57:45.516714 17462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:45.516736 17462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:45.516785 17462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:45.517266 17462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "931c15f4758f4b0d96451849bcd45e03" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33789 } }
I20260619 16:57:45.517405 17462 leader_election.cc:302] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 931c15f4758f4b0d96451849bcd45e03; no voters:
I20260619 16:57:45.517568 17462 leader_election.cc:288] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:45.517656 17465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:45.517818 17462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:45.517818 17465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [term 1 LEADER]: Becoming Leader. State: Replica: 931c15f4758f4b0d96451849bcd45e03, State: Running, Role: LEADER
I20260619 16:57:45.517977 17465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "931c15f4758f4b0d96451849bcd45e03" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33789 } }
I20260619 16:57:45.518334 17467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 931c15f4758f4b0d96451849bcd45e03. Latest consensus state: current_term: 1 leader_uuid: "931c15f4758f4b0d96451849bcd45e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "931c15f4758f4b0d96451849bcd45e03" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33789 } } }
I20260619 16:57:45.518404 17466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "931c15f4758f4b0d96451849bcd45e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "931c15f4758f4b0d96451849bcd45e03" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33789 } } }
I20260619 16:57:45.518440 17467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:45.518497 17466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:45.519003 17474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:45.519467 17474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:45.520416 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 17395
I20260619 16:57:45.520527 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:45.520756 31314 external_mini_cluster.cc:1428] Setting key 20cef3e8375c7344325e478777c37a19
I20260619 16:57:45.521721 17474 catalog_manager.cc:1347] Generated new cluster ID: 510f54e9495d4e84a8887c87b952811b
I20260619 16:57:45.521772 17474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:45.526666 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.521194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.30.148.190:33789)
0619 16:57:45.521345 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.521375 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.521408 (+ 33us) server_negotiation.cc:407] Connection header received
0619 16:57:45.521518 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.521525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.521588 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.521719 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:45.522041 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.523459 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.524499 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.525122 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.525238 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.525903 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:45.525907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:45.525907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:45.526127 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:45.526166 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:45.526181 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:45.526246 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.526323 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.526411 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":64,"threads_started":1}
I20260619 16:57:45.540246 17474 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:45.540741 17474 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:45.545038 17474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03: Generated new TSK 0
I20260619 16:57:45.545222 17474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:57:45.551867 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33789
--builtin_ntp_servers=127.30.148.148:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 16:57:45.638509 17486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:45.638680 17486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:45.638700 17486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:45.639880 17486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:45.639918 17486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:45.639953 17486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:57:45.641269 17486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:45.641506 17486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:45.641795 17486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:45.646435 17491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:45.646446 17494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:45.646747 17492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:45.646770 17486 server_base.cc:1034] running on GCE node
I20260619 16:57:45.646983 17486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:45.647207 17486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:45.648360 17486 hybrid_clock.cc:648] HybridClock initialized: now 1781888265648346 us; error 26 us; skew 500 ppm
I20260619 16:57:45.649713 17486 webserver.cc:460] Webserver started at http://127.30.148.129:38097/ using document root <none> and password file <none>
I20260619 16:57:45.649968 17486 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:45.650022 17486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:45.650142 17486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:45.651491 17486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "353252392e1c47e1b7d18e5e18ec96bf"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "e6d110028ad667dfed5bc7cdaf3c346a"
server_key_iv: "5e9dbb65497003a78c37ff8ced848983"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.651914 17486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "353252392e1c47e1b7d18e5e18ec96bf"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "e6d110028ad667dfed5bc7cdaf3c346a"
server_key_iv: "5e9dbb65497003a78c37ff8ced848983"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.653443 17486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:57:45.654214 17500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.654392 17486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:45.654477 17486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "353252392e1c47e1b7d18e5e18ec96bf"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "e6d110028ad667dfed5bc7cdaf3c346a"
server_key_iv: "5e9dbb65497003a78c37ff8ced848983"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.654542 17486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:45.678524 17486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:45.678753 17486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:45.678865 17486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:45.679068 17486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:45.679384 17486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:45.679415 17486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.679464 17486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:45.679489 17486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.685032 17486 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:38749
I20260619 16:57:45.685077 17613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:38749 every 8 connection(s)
I20260619 16:57:45.685401 17486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:57:45.687191 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 17486
I20260619 16:57:45.687284 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:57:45.687466 31314 external_mini_cluster.cc:1428] Setting key ccfb3a28a0fc4df5c771ede785161e40
I20260619 16:57:45.688426 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33789
--builtin_ntp_servers=127.30.148.148:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 16:57:45.690786 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.686053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:56903 (local address 127.30.148.190:33789)
0619 16:57:45.686096 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.686099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.686334 (+ 235us) server_negotiation.cc:407] Connection header received
0619 16:57:45.686782 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.686783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.686793 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.686836 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:45.687570 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.688251 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.689261 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.689600 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.689674 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.690556 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:45.690557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:45.690558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:45.690561 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:45.690575 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:45.690578 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:45.690615 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.690708 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.690730 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:57:45.691051 17616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.686058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33789 (local address 127.30.148.129:56903)
0619 16:57:45.686228 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.686258 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.686696 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.686896 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.686899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.686907 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:45.687460 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.687465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.688335 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.688335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.689179 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.689181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.689688 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.689689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.689845 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.690398 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:57:45.690399 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:45.690437 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:57:45.690464 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:45.690631 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:45.690637 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:45.690639 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.690706 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.690808 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260619 16:57:45.691708 17614 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33789
I20260619 16:57:45.691802 17614 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:45.691990 17614 heartbeater.cc:511] Master 127.30.148.190:33789 requested a full tablet report, sending...
I20260619 16:57:45.692467 17426 ts_manager.cc:194] Registered new tserver with Master: 353252392e1c47e1b7d18e5e18ec96bf (127.30.148.129:38749)
I20260619 16:57:45.693027 17426 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.148.129:56903
W20260619 16:57:45.767679 17617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:45.767865 17617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:45.767886 17617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:45.769026 17617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:45.769064 17617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:45.769089 17617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:57:45.770448 17617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:45.770684 17617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:45.770956 17617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:45.775699 17622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:45.775748 17623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:45.775677 17625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:45.775899 17617 server_base.cc:1034] running on GCE node
I20260619 16:57:45.776062 17617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:45.776244 17617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:45.777423 17617 hybrid_clock.cc:648] HybridClock initialized: now 1781888265777413 us; error 26 us; skew 500 ppm
I20260619 16:57:45.778568 17617 webserver.cc:460] Webserver started at http://127.30.148.130:33107/ using document root <none> and password file <none>
I20260619 16:57:45.778795 17617 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:45.778839 17617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:45.778995 17617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:45.780047 17617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "c21408268b114be292bdff5f225a17f0"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "6a17c9b9bdda82397dd190551b91e4b8"
server_key_iv: "122276600b68ef13e85d47c0269a25fd"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.780465 17617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "c21408268b114be292bdff5f225a17f0"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "6a17c9b9bdda82397dd190551b91e4b8"
server_key_iv: "122276600b68ef13e85d47c0269a25fd"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.781767 17617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:57:45.782472 17631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.782634 17617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:57:45.782708 17617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "c21408268b114be292bdff5f225a17f0"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "6a17c9b9bdda82397dd190551b91e4b8"
server_key_iv: "122276600b68ef13e85d47c0269a25fd"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.782763 17617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:45.792495 17617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:45.792698 17617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:45.792789 17617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:45.792935 17617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:45.793212 17617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:45.793251 17617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.793270 17617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:45.793313 17617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.798633 17617 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:35865
I20260619 16:57:45.798703 17744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:35865 every 8 connection(s)
I20260619 16:57:45.798983 17617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:57:45.803289 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 17617
I20260619 16:57:45.803422 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:57:45.803622 31314 external_mini_cluster.cc:1428] Setting key 403de39397f0a81357fbba7f31bbce92
I20260619 16:57:45.803638 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.799642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:36979 (local address 127.30.148.190:33789)
0619 16:57:45.799702 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.799704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.799930 (+ 226us) server_negotiation.cc:407] Connection header received
0619 16:57:45.800349 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.800350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.800359 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.800411 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:45.800983 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.801505 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.802357 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.802528 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.802575 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.803325 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:45.803327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:45.803327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:45.803332 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:45.803347 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:45.803351 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:45.803425 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.803534 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.803563 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 16:57:45.803850 17747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.799620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33789 (local address 127.30.148.130:36979)
0619 16:57:45.799798 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.799834 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.800270 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.800456 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.800459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.800466 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:45.800918 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.800921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.801571 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.801572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.802290 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.802292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.802591 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.802592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.802687 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.803182 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:57:45.803183 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:45.803207 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:57:45.803226 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:45.803457 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:45.803462 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:45.803465 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.803533 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.803623 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":85,"threads_started":1}
I20260619 16:57:45.804335 17745 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33789
I20260619 16:57:45.804461 17745 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:45.804636 17745 heartbeater.cc:511] Master 127.30.148.190:33789 requested a full tablet report, sending...
I20260619 16:57:45.804682 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33789
--builtin_ntp_servers=127.30.148.148:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 16:57:45.805094 17426 ts_manager.cc:194] Registered new tserver with Master: c21408268b114be292bdff5f225a17f0 (127.30.148.130:35865)
I20260619 16:57:45.805521 17426 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.148.130:36979
W20260619 16:57:45.882930 17748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:45.883107 17748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:45.883128 17748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:45.884191 17748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:45.884222 17748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:45.884245 17748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:57:45.885690 17748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:33789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:45.885951 17748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:45.886281 17748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:45.891012 17756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:45.891019 17754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:45.891036 17748 server_base.cc:1034] running on GCE node
W20260619 16:57:45.891033 17753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:45.891391 17748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:45.891635 17748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:45.892788 17748 hybrid_clock.cc:648] HybridClock initialized: now 1781888265892776 us; error 28 us; skew 500 ppm
I20260619 16:57:45.893946 17748 webserver.cc:460] Webserver started at http://127.30.148.131:37019/ using document root <none> and password file <none>
I20260619 16:57:45.894177 17748 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:45.894227 17748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:45.894387 17748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:45.895370 17748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "439d3c7349e847369a59e1414c5101ac"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "dbdc23e3489fbfd04c4626b0bc536cc0"
server_key_iv: "41241c763ba3c1d773bc682af1a13fa2"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.895694 17748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "439d3c7349e847369a59e1414c5101ac"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "dbdc23e3489fbfd04c4626b0bc536cc0"
server_key_iv: "41241c763ba3c1d773bc682af1a13fa2"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.896919 17748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:57:45.897634 17762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.897799 17748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:57:45.897892 17748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "439d3c7349e847369a59e1414c5101ac"
format_stamp: "Formatted at 2026-06-19 16:57:45 on dist-test-slave-xb9n"
server_key: "dbdc23e3489fbfd04c4626b0bc536cc0"
server_key_iv: "41241c763ba3c1d773bc682af1a13fa2"
server_key_version: "encryptionkey@0"
I20260619 16:57:45.897964 17748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:45.908629 17748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:45.908876 17748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:45.909005 17748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:45.909226 17748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:45.909576 17748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:45.909626 17748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.909659 17748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:45.909686 17748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:45.914919 17748 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:45229
I20260619 16:57:45.914988 17875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:45229 every 8 connection(s)
I20260619 16:57:45.915287 17748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:57:45.919486 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 17748
I20260619 16:57:45.919570 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:57:45.919775 31314 external_mini_cluster.cc:1428] Setting key f1f609c962b595fa666c0c9a967946ea
I20260619 16:57:45.920466 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.915957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:42605 (local address 127.30.148.190:33789)
0619 16:57:45.916030 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.916032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.916255 (+ 223us) server_negotiation.cc:407] Connection header received
0619 16:57:45.916737 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.916738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.916749 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.916796 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:45.917445 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.918062 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.919071 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.919274 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.919354 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.920197 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:45.920198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:45.920198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:45.920203 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:45.920218 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:45.920222 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:45.920284 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.920363 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.920392 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 16:57:45.920691 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.915923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:33789 (local address 127.30.148.131:42605)
0619 16:57:45.916135 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.916174 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.916640 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.916876 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.916880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.916893 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:45.917336 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.917340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.918147 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.918148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.918981 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.918984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.919360 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.919361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.919475 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.920057 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:57:45.920057 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:45.920097 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:57:45.920119 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:45.920285 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:45.920291 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:45.920294 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.920365 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.920458 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260619 16:57:45.921079 17876 heartbeater.cc:347] Connected to a master server at 127.30.148.190:33789
I20260619 16:57:45.921177 17876 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:45.921343 17876 heartbeater.cc:511] Master 127.30.148.190:33789 requested a full tablet report, sending...
I20260619 16:57:45.921690 17426 ts_manager.cc:194] Registered new tserver with Master: 439d3c7349e847369a59e1414c5101ac (127.30.148.131:45229)
I20260619 16:57:45.922039 17426 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.148.131:42605
I20260619 16:57:45.931627 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:57:45.935643 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.932880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51234 (local address 127.30.148.190:33789)
0619 16:57:45.932947 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.932949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.933023 (+ 74us) server_negotiation.cc:407] Connection header received
0619 16:57:45.933100 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.933101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.933112 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.933173 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:45.933322 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.933812 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.934556 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.934735 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.934812 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.935374 (+ 562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:45.935375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:45.935376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:45.935380 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:45.935391 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:45.935395 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:45.935495 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.935557 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.935566 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 16:57:45.937011 17426 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:57:45.938500 17426 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:57:45.944187 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.940367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:35865 (local address 127.0.0.1:41946)
0619 16:57:45.940775 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.940785 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.940839 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.941058 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.941059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.941061 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:45.941106 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.941107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.942052 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.942053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.943236 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.943239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.943905 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.943906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.943985 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.943998 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.944069 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.944120 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":91,"threads_started":1}
I20260619 16:57:45.944200 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.940599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41946 (local address 127.30.148.130:35865)
0619 16:57:45.940773 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.940777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.940844 (+ 67us) server_negotiation.cc:407] Connection header received
0619 16:57:45.940907 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.940909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.940924 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.940996 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:45.941182 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.941972 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.943305 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.943817 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.943905 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.943989 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.944082 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.944121 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260619 16:57:45.944401 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.940321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:45229 (local address 127.0.0.1:34198)
0619 16:57:45.940416 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.940449 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.940526 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.940992 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.940992 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.940994 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:45.941042 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.941044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.942343 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.942343 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.943336 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.943338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.944207 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.944208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.944268 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.944271 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.944298 (+ 27us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.944334 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":57,"threads_started":1}
I20260619 16:57:45.944748 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.940518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34198 (local address 127.30.148.131:45229)
0619 16:57:45.940843 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.940848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.940857 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:57:45.940892 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.940894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.940907 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.940994 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:45.941120 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.942259 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.943403 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.944110 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.944511 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.944614 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.944639 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.944674 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":72,"threads_started":1}
I20260619 16:57:45.945086 17679 tablet_service.cc:1467] Processing CreateTablet for tablet e749d6287caa4570b0f3955f42a85ae6 (DEFAULT_TABLE table=test-table [id=1c77b47fb0aa4470b902933c654e2e6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:45.945291 17810 tablet_service.cc:1467] Processing CreateTablet for tablet e749d6287caa4570b0f3955f42a85ae6 (DEFAULT_TABLE table=test-table [id=1c77b47fb0aa4470b902933c654e2e6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:45.945472 17679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e749d6287caa4570b0f3955f42a85ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:45.945657 17810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e749d6287caa4570b0f3955f42a85ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:45.945803 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.940362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:38749 (local address 127.0.0.1:34582)
0619 16:57:45.940666 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:45.940676 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:45.940726 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:45.940962 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:45.940964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:45.940969 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:45.941024 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.941026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.943421 (+ 2395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.943421 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.944558 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:45.944561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.945563 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.945564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:45.945630 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.945634 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:57:45.945696 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 16:57:45.945734 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":79,"threads_started":1}
I20260619 16:57:45.946028 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:45.940554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.30.148.129:38749)
0619 16:57:45.940716 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:45.940721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:45.940732 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:57:45.940794 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:45.940796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:45.940810 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:45.940880 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:45.941093 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.942261 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.944637 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:45.945379 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:45.945449 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:45.945529 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:45.945762 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 16:57:45.945812 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260619 16:57:45.946767 17548 tablet_service.cc:1467] Processing CreateTablet for tablet e749d6287caa4570b0f3955f42a85ae6 (DEFAULT_TABLE table=test-table [id=1c77b47fb0aa4470b902933c654e2e6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:45.947095 17548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e749d6287caa4570b0f3955f42a85ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:45.948063 17890 tablet_bootstrap.cc:492] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac: Bootstrap starting.
I20260619 16:57:45.948635 17890 tablet_bootstrap.cc:654] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:45.948737 17891 tablet_bootstrap.cc:492] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0: Bootstrap starting.
I20260619 16:57:45.948940 17890 log.cc:824] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:45.949314 17891 tablet_bootstrap.cc:654] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:45.949642 17891 log.cc:824] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:45.949726 17890 tablet_bootstrap.cc:492] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac: No bootstrap required, opened a new log
I20260619 16:57:45.949780 17890 ts_tablet_manager.cc:1397] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:57:45.950193 17894 tablet_bootstrap.cc:492] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf: Bootstrap starting.
I20260619 16:57:45.950363 17891 tablet_bootstrap.cc:492] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0: No bootstrap required, opened a new log
I20260619 16:57:45.950433 17891 ts_tablet_manager.cc:1397] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 16:57:45.950728 17894 tablet_bootstrap.cc:654] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:45.951005 17894 log.cc:824] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:45.951689 17894 tablet_bootstrap.cc:492] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf: No bootstrap required, opened a new log
I20260619 16:57:45.951830 17894 ts_tablet_manager.cc:1397] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:57:45.952421 17890 raft_consensus.cc:348] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439d3c7349e847369a59e1414c5101ac" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45229 } } peers { permanent_uuid: "353252392e1c47e1b7d18e5e18ec96bf" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38749 } } peers { permanent_uuid: "c21408268b114be292bdff5f225a17f0" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35865 } }
I20260619 16:57:45.952582 17890 raft_consensus.cc:374] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:45.952617 17890 raft_consensus.cc:729] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 439d3c7349e847369a59e1414c5101ac, State: Initialized, Role: FOLLOWER
I20260619 16:57:45.952755 17890 consensus_queue.cc:260] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439d3c7349e847369a59e1414c5101ac" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45229 } } peers { permanent_uuid: "353252392e1c47e1b7d18e5e18ec96bf" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38749 } } peers { permanent_uuid: "c21408268b114be292bdff5f225a17f0" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35865 } }
I20260619 16:57:45.952993 17890 ts_tablet_manager.cc:1428] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:57:45.952937 17891 raft_consensus.cc:348] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439d3c7349e847369a59e1414c5101ac" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45229 } } peers { permanent_uuid: "353252392e1c47e1b7d18e5e18ec96bf" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38749 } } peers { permanent_uuid: "c21408268b114be292bdff5f225a17f0" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35865 } }
I20260619 16:57:45.953078 17891 raft_consensus.cc:374] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:45.953112 17876 heartbeater.cc:503] Master 127.30.148.190:33789 was elected leader, sending a full tablet report...
I20260619 16:57:45.953120 17891 raft_consensus.cc:729] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c21408268b114be292bdff5f225a17f0, State: Initialized, Role: FOLLOWER
I20260619 16:57:45.953225 17891 consensus_queue.cc:260] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439d3c7349e847369a59e1414c5101ac" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45229 } } peers { permanent_uuid: "353252392e1c47e1b7d18e5e18ec96bf" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38749 } } peers { permanent_uuid: "c21408268b114be292bdff5f225a17f0" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35865 } }
I20260619 16:57:45.953471 17891 ts_tablet_manager.cc:1428] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 16:57:45.953591 17745 heartbeater.cc:503] Master 127.30.148.190:33789 was elected leader, sending a full tablet report...
I20260619 16:57:45.954456 17894 raft_consensus.cc:348] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439d3c7349e847369a59e1414c5101ac" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45229 } } peers { permanent_uuid: "353252392e1c47e1b7d18e5e18ec96bf" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38749 } } peers { permanent_uuid: "c21408268b114be292bdff5f225a17f0" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35865 } }
I20260619 16:57:45.954600 17894 raft_consensus.cc:374] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:45.954643 17894 raft_consensus.cc:729] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353252392e1c47e1b7d18e5e18ec96bf, State: Initialized, Role: FOLLOWER
I20260619 16:57:45.954744 17894 consensus_queue.cc:260] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439d3c7349e847369a59e1414c5101ac" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45229 } } peers { permanent_uuid: "353252392e1c47e1b7d18e5e18ec96bf" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38749 } } peers { permanent_uuid: "c21408268b114be292bdff5f225a17f0" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35865 } }
I20260619 16:57:45.954968 17894 ts_tablet_manager.cc:1428] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 16:57:45.955052 17614 heartbeater.cc:503] Master 127.30.148.190:33789 was elected leader, sending a full tablet report...
I20260619 16:57:46.041173 17896 raft_consensus.cc:482] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:57:46.041278 17896 raft_consensus.cc:504] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439d3c7349e847369a59e1414c5101ac" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45229 } } peers { permanent_uuid: "353252392e1c47e1b7d18e5e18ec96bf" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38749 } } peers { permanent_uuid: "c21408268b114be292bdff5f225a17f0" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35865 } }
I20260619 16:57:46.041736 17896 leader_election.cc:288] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 353252392e1c47e1b7d18e5e18ec96bf (127.30.148.129:38749), c21408268b114be292bdff5f225a17f0 (127.30.148.130:35865)
I20260619 16:57:46.044440 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:46.041894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:38749 (local address 127.30.148.131:45115)
0619 16:57:46.041936 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:46.041948 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:46.041994 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:46.042154 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:46.042169 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:46.042171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:46.042243 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:46.042245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:46.042887 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:46.042888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:46.043667 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:46.043670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:46.044199 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:46.044200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:46.044270 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:46.044274 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:57:46.044343 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 16:57:46.044390 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":25}
I20260619 16:57:46.044479 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:46.041960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:45115 (local address 127.30.148.129:38749)
0619 16:57:46.042004 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:46.042006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:46.042018 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:57:46.042063 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:46.042064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:46.042076 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:46.042145 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:46.042313 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:46.042814 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:46.043742 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:46.044124 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:46.044184 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:46.044190 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:46.044355 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 16:57:46.044390 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:57:46.044616 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:46.041923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:35865 (local address 127.30.148.131:56327)
0619 16:57:46.042083 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:46.042124 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:46.042175 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:46.042313 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:46.042314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:46.042315 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:46.042379 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:46.042380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:46.042948 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:46.042949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:46.043783 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:46.043785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:46.044339 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:46.044340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:46.044424 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:46.044429 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:57:46.044484 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 16:57:46.044523 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260619 16:57:46.044618 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:46.042020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:56327 (local address 127.30.148.130:35865)
0619 16:57:46.042062 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:46.042064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:46.042180 (+ 116us) server_negotiation.cc:407] Connection header received
0619 16:57:46.042228 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:46.042229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:46.042238 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:46.042291 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:46.042441 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:46.042875 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:46.043860 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:46.044229 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:46.044315 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:46.044324 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:46.044500 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 16:57:46.044541 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:57:46.044780 17568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e749d6287caa4570b0f3955f42a85ae6" candidate_uuid: "439d3c7349e847369a59e1414c5101ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353252392e1c47e1b7d18e5e18ec96bf" is_pre_election: true
I20260619 16:57:46.044878 17699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e749d6287caa4570b0f3955f42a85ae6" candidate_uuid: "439d3c7349e847369a59e1414c5101ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c21408268b114be292bdff5f225a17f0" is_pre_election: true
I20260619 16:57:46.044927 17568 raft_consensus.cc:2454] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 439d3c7349e847369a59e1414c5101ac in term 0.
I20260619 16:57:46.044996 17699 raft_consensus.cc:2454] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 439d3c7349e847369a59e1414c5101ac in term 0.
I20260619 16:57:46.045142 17763 leader_election.cc:302] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353252392e1c47e1b7d18e5e18ec96bf, 439d3c7349e847369a59e1414c5101ac; no voters:
I20260619 16:57:46.045258 17896 raft_consensus.cc:2781] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:57:46.045310 17896 raft_consensus.cc:482] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:57:46.045333 17896 raft_consensus.cc:3037] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:46.046058 17896 raft_consensus.cc:504] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439d3c7349e847369a59e1414c5101ac" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45229 } } peers { permanent_uuid: "353252392e1c47e1b7d18e5e18ec96bf" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38749 } } peers { permanent_uuid: "c21408268b114be292bdff5f225a17f0" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35865 } }
I20260619 16:57:46.046200 17896 leader_election.cc:288] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [CANDIDATE]: Term 1 election: Requested vote from peers 353252392e1c47e1b7d18e5e18ec96bf (127.30.148.129:38749), c21408268b114be292bdff5f225a17f0 (127.30.148.130:35865)
I20260619 16:57:46.046357 17568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e749d6287caa4570b0f3955f42a85ae6" candidate_uuid: "439d3c7349e847369a59e1414c5101ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353252392e1c47e1b7d18e5e18ec96bf"
I20260619 16:57:46.046358 17699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e749d6287caa4570b0f3955f42a85ae6" candidate_uuid: "439d3c7349e847369a59e1414c5101ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c21408268b114be292bdff5f225a17f0"
I20260619 16:57:46.046427 17568 raft_consensus.cc:3037] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:46.046427 17699 raft_consensus.cc:3037] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:46.047261 17568 raft_consensus.cc:2454] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 439d3c7349e847369a59e1414c5101ac in term 1.
I20260619 16:57:46.047261 17699 raft_consensus.cc:2454] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 439d3c7349e847369a59e1414c5101ac in term 1.
I20260619 16:57:46.047447 17766 leader_election.cc:302] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 439d3c7349e847369a59e1414c5101ac, c21408268b114be292bdff5f225a17f0; no voters:
I20260619 16:57:46.047533 17896 raft_consensus.cc:2781] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:46.047676 17896 raft_consensus.cc:686] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 1 LEADER]: Becoming Leader. State: Replica: 439d3c7349e847369a59e1414c5101ac, State: Running, Role: LEADER
I20260619 16:57:46.047770 17896 consensus_queue.cc:237] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439d3c7349e847369a59e1414c5101ac" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45229 } } peers { permanent_uuid: "353252392e1c47e1b7d18e5e18ec96bf" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38749 } } peers { permanent_uuid: "c21408268b114be292bdff5f225a17f0" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35865 } }
I20260619 16:57:46.048520 17426 catalog_manager.cc:5495] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 439d3c7349e847369a59e1414c5101ac (127.30.148.131). New cstate: current_term: 1 leader_uuid: "439d3c7349e847369a59e1414c5101ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439d3c7349e847369a59e1414c5101ac" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 45229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "353252392e1c47e1b7d18e5e18ec96bf" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c21408268b114be292bdff5f225a17f0" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35865 } health_report { overall_health: UNKNOWN } } }
W20260619 16:57:46.049458 17746 tablet.cc:2369] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:57:46.061714 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:46.059513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34204 (local address 127.30.148.131:45229)
0619 16:57:46.059557 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:46.059558 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:46.059567 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:57:46.059611 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:46.059612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:46.059633 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:46.059703 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:57:46.059805 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:46.060227 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:46.060931 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:46.061103 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:46.061166 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:46.061329 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:57:46.061478 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:57:46.061590 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:46.061621 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 16:57:46.061628 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:57:46.063258 17568 raft_consensus.cc:1261] T e749d6287caa4570b0f3955f42a85ae6 P 353252392e1c47e1b7d18e5e18ec96bf [term 1 FOLLOWER]: Refusing update from remote peer 439d3c7349e847369a59e1414c5101ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:46.063270 17699 raft_consensus.cc:1261] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0 [term 1 FOLLOWER]: Refusing update from remote peer 439d3c7349e847369a59e1414c5101ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:46.063609 17900 consensus_queue.cc:1035] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "c21408268b114be292bdff5f225a17f0" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 35865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:46.063714 17896 consensus_queue.cc:1035] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "353252392e1c47e1b7d18e5e18ec96bf" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:46.066118 17904 mvcc.cc:205] Tried to move back new op lower bound from 7298614337792901120 to 7298614337732321280. Current Snapshot: MvccSnapshot[applied={T|T < 7298614337792901120}]
I20260619 16:57:46.068213 17426 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:46.068303 17426 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:46.068980 17426 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 931c15f4758f4b0d96451849bcd45e03: Sending DeleteTablet for 3 replicas of tablet e749d6287caa4570b0f3955f42a85ae6
I20260619 16:57:46.069303 17548 tablet_service.cc:1514] Processing DeleteTablet for tablet e749d6287caa4570b0f3955f42a85ae6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:46 UTC) from {username='slave'} at 127.0.0.1:34582
I20260619 16:57:46.069312 17679 tablet_service.cc:1514] Processing DeleteTablet for tablet e749d6287caa4570b0f3955f42a85ae6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:46 UTC) from {username='slave'} at 127.0.0.1:41946
I20260619 16:57:46.069561 17914 tablet_replica.cc:330] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0: stopping tablet replica
I20260619 16:57:46.069654 17914 raft_consensus.cc:2229] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:46.069751 17914 raft_consensus.cc:2258] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:46.069753 17810 tablet_service.cc:1514] Processing DeleteTablet for tablet e749d6287caa4570b0f3955f42a85ae6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:46 UTC) from {username='slave'} at 127.0.0.1:34198
I20260619 16:57:46.069857 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 17486
I20260619 16:57:46.070623 17914 ts_tablet_manager.cc:1905] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:46.072602 17914 ts_tablet_manager.cc:1918] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:46.072734 17914 log.cc:1196] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/e749d6287caa4570b0f3955f42a85ae6
I20260619 16:57:46.073035 17914 ts_tablet_manager.cc:1939] T e749d6287caa4570b0f3955f42a85ae6 P c21408268b114be292bdff5f225a17f0: Deleting consensus metadata
I20260619 16:57:46.073808 17413 catalog_manager.cc:4841] TS c21408268b114be292bdff5f225a17f0 (127.30.148.130:35865): tablet e749d6287caa4570b0f3955f42a85ae6 (table test-table [id=1c77b47fb0aa4470b902933c654e2e6b]) successfully deleted
I20260619 16:57:46.077456 17915 tablet_replica.cc:330] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac: stopping tablet replica
I20260619 16:57:46.077553 17915 raft_consensus.cc:2229] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:57:46.077685 17915 raft_consensus.cc:2258] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:46.078078 17915 ts_tablet_manager.cc:1905] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:46.079500 17915 ts_tablet_manager.cc:1918] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:46.079571 17915 log.cc:1196] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/e749d6287caa4570b0f3955f42a85ae6
I20260619 16:57:46.079828 17915 ts_tablet_manager.cc:1939] T e749d6287caa4570b0f3955f42a85ae6 P 439d3c7349e847369a59e1414c5101ac: Deleting consensus metadata
I20260619 16:57:46.080175 17412 catalog_manager.cc:4841] TS 439d3c7349e847369a59e1414c5101ac (127.30.148.131:45229): tablet e749d6287caa4570b0f3955f42a85ae6 (table test-table [id=1c77b47fb0aa4470b902933c654e2e6b]) successfully deleted
W20260619 16:57:46.080753 17410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:38749 (error 108)
I20260619 16:57:46.081010 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 17617
I20260619 16:57:46.082495 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:46.082362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:38749 (local address 127.0.0.1:34588)
0619 16:57:46.082414 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:46.082463 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:38749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 16:57:46.082614 17410 catalog_manager.cc:4584] TS 353252392e1c47e1b7d18e5e18ec96bf (127.30.148.129:38749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e749d6287caa4570b0f3955f42a85ae6: Network error: Client connection negotiation failed: client connection to 127.30.148.129:38749: connect: Connection refused (error 111)
I20260619 16:57:46.085932 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 17748
I20260619 16:57:46.089802 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 17395
2026-06-19T16:57:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (728 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:57:46 dist-test-slave-xb9n krb5kdc[17919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:57:46 dist-test-slave-xb9n krb5kdc[17919](info): set up 2 sockets
Jun 19 16:57:46 dist-test-slave-xb9n krb5kdc[17919](info): commencing operation
krb5kdc: starting...
W20260619 16:57:47.113173 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:57:47 dist-test-slave-xb9n krb5kdc[17919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:57:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:57:47.223822 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:38129
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:38129
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:47.302407 17932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:47.302618 17932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:47.302651 17932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:47.303732 17932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:47.303792 17932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:47.303817 17932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:47.303844 17932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:47.303874 17932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:47.305279 17932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:38129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:38129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:47.305545 17932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:47.305747 17932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:47.310456 17940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:47.310593 17932 server_base.cc:1034] running on GCE node
W20260619 16:57:47.310451 17938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:47.310451 17937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:47.310966 17932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:47.311220 17932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:47.312354 17932 hybrid_clock.cc:648] HybridClock initialized: now 1781888267312336 us; error 35 us; skew 500 ppm
Jun 19 16:57:47 dist-test-slave-xb9n krb5kdc[17919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:47.314635 17932 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:57:47.315042 17932 webserver.cc:460] Webserver started at http://127.30.148.190:38949/ using document root <none> and password file <none>
I20260619 16:57:47.315208 17932 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:47.315246 17932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:47.315327 17932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:47.316236 17932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "735ec4efe405491ebcaa501f94373a28"
format_stamp: "Formatted at 2026-06-19 16:57:47 on dist-test-slave-xb9n"
server_key: "d5c4812771af7f5b08426d62d2652104"
server_key_iv: "3adafe93b731ee706265afc02a6a86aa"
server_key_version: "encryptionkey@0"
I20260619 16:57:47.316519 17932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "735ec4efe405491ebcaa501f94373a28"
format_stamp: "Formatted at 2026-06-19 16:57:47 on dist-test-slave-xb9n"
server_key: "d5c4812771af7f5b08426d62d2652104"
server_key_iv: "3adafe93b731ee706265afc02a6a86aa"
server_key_version: "encryptionkey@0"
I20260619 16:57:47.317765 17932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:57:47.318413 17947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:47.318610 17932 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:57:47.318677 17932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "735ec4efe405491ebcaa501f94373a28"
format_stamp: "Formatted at 2026-06-19 16:57:47 on dist-test-slave-xb9n"
server_key: "d5c4812771af7f5b08426d62d2652104"
server_key_iv: "3adafe93b731ee706265afc02a6a86aa"
server_key_version: "encryptionkey@0"
I20260619 16:57:47.318727 17932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:47.346673 17932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:47.348404 17932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:47.348508 17932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:47.352013 17932 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:38129
I20260619 16:57:47.352063 17999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:38129 every 8 connection(s)
I20260619 16:57:47.352412 17932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:47.353204 18000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:47.355273 18000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28: Bootstrap starting.
I20260619 16:57:47.355779 18000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:47.356003 18000 log.cc:824] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:47.356619 18000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28: No bootstrap required, opened a new log
I20260619 16:57:47.357988 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 17932
I20260619 16:57:47.358086 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:47.358294 31314 external_mini_cluster.cc:1428] Setting key ffeeab0d5b85557122684748f84f0b2e
I20260619 16:57:47.358268 18000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735ec4efe405491ebcaa501f94373a28" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38129 } }
I20260619 16:57:47.358431 18000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:47.358479 18000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735ec4efe405491ebcaa501f94373a28, State: Initialized, Role: FOLLOWER
I20260619 16:57:47.358609 18000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735ec4efe405491ebcaa501f94373a28" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38129 } }
I20260619 16:57:47.358687 18000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:47.358731 18000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:47.358772 18000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:47.359335 18000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735ec4efe405491ebcaa501f94373a28" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38129 } }
I20260619 16:57:47.359449 18000 leader_election.cc:302] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 735ec4efe405491ebcaa501f94373a28; no voters:
I20260619 16:57:47.359606 18000 leader_election.cc:288] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:47.359638 18005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:47.359740 18005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [term 1 LEADER]: Becoming Leader. State: Replica: 735ec4efe405491ebcaa501f94373a28, State: Running, Role: LEADER
I20260619 16:57:47.359838 18000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:47.359813 18005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735ec4efe405491ebcaa501f94373a28" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38129 } }
I20260619 16:57:47.360136 18005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "735ec4efe405491ebcaa501f94373a28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735ec4efe405491ebcaa501f94373a28" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38129 } } }
I20260619 16:57:47.360208 18005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:47.360325 18005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 735ec4efe405491ebcaa501f94373a28. Latest consensus state: current_term: 1 leader_uuid: "735ec4efe405491ebcaa501f94373a28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735ec4efe405491ebcaa501f94373a28" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38129 } } }
I20260619 16:57:47.360395 18005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:47.360558 18010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:47.361070 18010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:47.362629 18010 catalog_manager.cc:1347] Generated new cluster ID: 7bbcfb2fd89941698b07553c2ab13f7e
I20260619 16:57:47.362680 18010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:57:47 dist-test-slave-xb9n krb5kdc[17919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:47.369899 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.358769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.30.148.190:38129)
0619 16:57:47.358967 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:47.358976 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:47.359009 (+ 33us) server_negotiation.cc:407] Connection header received
0619 16:57:47.359317 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:47.359322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:47.359364 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:47.359495 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:47.360341 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.361901 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.363002 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.363603 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.363715 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.366402 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:47.366405 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:47.366406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:47.366417 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:47.368307 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:47.368707 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:47.368710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:47.368710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:47.368823 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:47.369027 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:47.369028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:47.369028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:47.369297 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:47.369443 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:47.369585 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 16:57:47.369708 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260619 16:57:47.376478 18010 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:47.376951 18010 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:47.381031 18010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28: Generated new TSK 0
I20260619 16:57:47.381199 18010 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.148.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
Entry for principal kudu/127.30.148.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab.
I20260619 16:57:47.409754 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38129
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:47.487463 18026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:47.487664 18026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:47.487699 18026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:47.488811 18026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:47.488870 18026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:47.488915 18026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:57:47.490389 18026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.129.keytab
--principal=kudu/127.30.148.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:47.490638 18026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:47.490840 18026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:47.495594 18034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:47.495632 18031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:47.495604 18032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:47.495828 18026 server_base.cc:1034] running on GCE node
I20260619 16:57:47.496057 18026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:47.496306 18026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:47.497414 18026 hybrid_clock.cc:648] HybridClock initialized: now 1781888267497402 us; error 37 us; skew 500 ppm
Jun 19 16:57:47 dist-test-slave-xb9n krb5kdc[17919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:47.500372 18026 init.cc:377] Logged in from keytab as kudu/127.30.148.129@KRBTEST.COM (short username kudu)
I20260619 16:57:47.500880 18026 webserver.cc:460] Webserver started at http://127.30.148.129:35765/ using document root <none> and password file <none>
I20260619 16:57:47.501204 18026 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:47.501266 18026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:47.501428 18026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:47.502501 18026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "19714ce9cfe04c9391ae23488ae10fa5"
format_stamp: "Formatted at 2026-06-19 16:57:47 on dist-test-slave-xb9n"
server_key: "4dc8a870b645dca8091b3327992881f1"
server_key_iv: "48eda22bc9b406407e0ba5199d6b8fe3"
server_key_version: "encryptionkey@0"
I20260619 16:57:47.502837 18026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "19714ce9cfe04c9391ae23488ae10fa5"
format_stamp: "Formatted at 2026-06-19 16:57:47 on dist-test-slave-xb9n"
server_key: "4dc8a870b645dca8091b3327992881f1"
server_key_iv: "48eda22bc9b406407e0ba5199d6b8fe3"
server_key_version: "encryptionkey@0"
I20260619 16:57:47.504153 18026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:57:47.504808 18041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:47.504969 18026 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:47.505049 18026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "19714ce9cfe04c9391ae23488ae10fa5"
format_stamp: "Formatted at 2026-06-19 16:57:47 on dist-test-slave-xb9n"
server_key: "4dc8a870b645dca8091b3327992881f1"
server_key_iv: "48eda22bc9b406407e0ba5199d6b8fe3"
server_key_version: "encryptionkey@0"
I20260619 16:57:47.505116 18026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:47.519726 18026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:47.521590 18026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:47.521729 18026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:47.521934 18026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:47.522266 18026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:47.522301 18026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:47.522337 18026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:47.522362 18026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:47.527863 18026 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:34703
I20260619 16:57:47.527909 18154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:34703 every 8 connection(s)
I20260619 16:57:47.528199 18026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
Jun 19 16:57:47 dist-test-slave-xb9n krb5kdc[17919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.129@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:47.534193 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 18026
I20260619 16:57:47.534312 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:57:47.534518 31314 external_mini_cluster.cc:1428] Setting key 67e2825a9c6ff6822331190db302abdb
I20260619 16:57:47.537473 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.528957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:54329 (local address 127.30.148.190:38129)
0619 16:57:47.529005 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:47.529009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:47.529206 (+ 197us) server_negotiation.cc:407] Connection header received
0619 16:57:47.529641 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:47.529642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:47.529654 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:47.529707 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:47.530549 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.531044 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.532041 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.532230 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.532292 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.534403 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:47.534404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:47.534405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:47.534412 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:47.536087 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:47.536458 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:47.536459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:47.536459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:47.536522 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:47.536829 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:47.536829 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:47.536830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:47.536927 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:47.536978 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:47.537280 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 16:57:47.537341 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:57:47.537671 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.528946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:38129 (local address 127.30.148.129:54329)
0619 16:57:47.529094 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:47.529131 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:47.529565 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:47.529769 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:47.529773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:47.530307 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:47.530479 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.530483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.531127 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.531128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.531960 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.531963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.532308 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.532308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.532400 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.532866 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:47.532873 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:47.534277 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:47.536172 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:47.536178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:47.536180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:47.536388 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:47.536595 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:47.536596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:47.536597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:47.536764 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:47.536987 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:47.536992 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:47.537147 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 16:57:47.537262 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 16:57:47.537446 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260619 16:57:47.538173 18155 heartbeater.cc:347] Connected to a master server at 127.30.148.190:38129
I20260619 16:57:47.538291 18155 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:47.538468 18155 heartbeater.cc:511] Master 127.30.148.190:38129 requested a full tablet report, sending...
I20260619 16:57:47.538961 17964 ts_manager.cc:194] Registered new tserver with Master: 19714ce9cfe04c9391ae23488ae10fa5 (127.30.148.129:34703)
I20260619 16:57:47.539546 17964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.129@KRBTEST.COM'} at 127.30.148.129:54329
WARNING: no policy specified for kudu/127.30.148.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
Entry for principal kudu/127.30.148.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab.
I20260619 16:57:47.559875 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38129
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:47.637123 18160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:47.637295 18160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:47.637316 18160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:47.638381 18160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:47.638418 18160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:47.638443 18160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:57:47.639734 18160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.130.keytab
--principal=kudu/127.30.148.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:47.639889 18160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:47.640033 18160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:57:47.644649 18160 server_base.cc:1034] running on GCE node
W20260619 16:57:47.644556 18168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:47.644569 18166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:47.644557 18165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:47.644975 18160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:47.645215 18160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:47.646349 18160 hybrid_clock.cc:648] HybridClock initialized: now 1781888267646336 us; error 29 us; skew 500 ppm
Jun 19 16:57:47 dist-test-slave-xb9n krb5kdc[17919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:47.648608 18160 init.cc:377] Logged in from keytab as kudu/127.30.148.130@KRBTEST.COM (short username kudu)
I20260619 16:57:47.648958 18160 webserver.cc:460] Webserver started at http://127.30.148.130:43913/ using document root <none> and password file <none>
I20260619 16:57:47.649127 18160 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:47.649166 18160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:47.649250 18160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:47.650233 18160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "b75c8563fa80428d92d2403e86ff87c1"
format_stamp: "Formatted at 2026-06-19 16:57:47 on dist-test-slave-xb9n"
server_key: "489237227f34e18a30ab0a4db3b6c493"
server_key_iv: "7723fac8ea8f2948abd8c416b42a0e49"
server_key_version: "encryptionkey@0"
I20260619 16:57:47.650511 18160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "b75c8563fa80428d92d2403e86ff87c1"
format_stamp: "Formatted at 2026-06-19 16:57:47 on dist-test-slave-xb9n"
server_key: "489237227f34e18a30ab0a4db3b6c493"
server_key_iv: "7723fac8ea8f2948abd8c416b42a0e49"
server_key_version: "encryptionkey@0"
I20260619 16:57:47.651664 18160 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:57:47.652281 18175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:47.652442 18160 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:47.652532 18160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "b75c8563fa80428d92d2403e86ff87c1"
format_stamp: "Formatted at 2026-06-19 16:57:47 on dist-test-slave-xb9n"
server_key: "489237227f34e18a30ab0a4db3b6c493"
server_key_iv: "7723fac8ea8f2948abd8c416b42a0e49"
server_key_version: "encryptionkey@0"
I20260619 16:57:47.652604 18160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:47.674607 18160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:47.676270 18160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:47.676416 18160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:47.676613 18160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:47.676908 18160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:47.676975 18160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:47.677011 18160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:47.677040 18160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:47.682530 18160 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:44917
I20260619 16:57:47.682578 18288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:44917 every 8 connection(s)
I20260619 16:57:47.682855 18160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:57:47.684063 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 18160
I20260619 16:57:47.684152 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:57:47.684370 31314 external_mini_cluster.cc:1428] Setting key 62b81d08551ecba01a812067999ceeb9
Jun 19 16:57:47 dist-test-slave-xb9n krb5kdc[17919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.130@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:47.692615 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.683518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:49103 (local address 127.30.148.190:38129)
0619 16:57:47.683596 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:47.683598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:47.683777 (+ 179us) server_negotiation.cc:407] Connection header received
0619 16:57:47.684382 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:47.684384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:47.684396 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:47.684447 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:47.685510 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.686003 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.686953 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.687295 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.687366 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.689807 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:47.689808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:47.689808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:47.689814 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:47.691173 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:47.691668 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:47.691668 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:47.691669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:47.691715 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:47.691974 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:47.691974 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:47.691974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:47.692099 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:47.692182 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:47.692492 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 16:57:47.692553 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 16:57:47.692884 18291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.683478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:38129 (local address 127.30.148.130:49103)
0619 16:57:47.683660 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:47.683698 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:47.684250 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:47.684512 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:47.684516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:47.685195 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:47.685413 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.685420 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.686093 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.686094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.686867 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.686869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.687382 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.687383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.687494 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.688028 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:47.688037 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:47.689720 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:47.691253 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:47.691260 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:47.691262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:47.691583 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:47.691773 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:47.691774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:47.691775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:47.691912 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:47.692187 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:47.692194 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:47.692347 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 16:57:47.692462 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 16:57:47.692641 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260619 16:57:47.693378 18289 heartbeater.cc:347] Connected to a master server at 127.30.148.190:38129
I20260619 16:57:47.693485 18289 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:47.693657 18289 heartbeater.cc:511] Master 127.30.148.190:38129 requested a full tablet report, sending...
I20260619 16:57:47.694010 17964 ts_manager.cc:194] Registered new tserver with Master: b75c8563fa80428d92d2403e86ff87c1 (127.30.148.130:44917)
WARNING: no policy specified for kudu/127.30.148.131@KRBTEST.COM; defaulting to no policy
I20260619 16:57:47.694341 17964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.130@KRBTEST.COM'} at 127.30.148.130:49103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
Entry for principal kudu/127.30.148.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab.
I20260619 16:57:47.710407 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38129
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:47.799741 18294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:47.799922 18294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:47.799943 18294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:47.801031 18294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:47.801071 18294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:47.801126 18294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:57:47.802562 18294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.131.keytab
--principal=kudu/127.30.148.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:38129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:47.802778 18294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:47.802986 18294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:47.807744 18300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:47.807757 18302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:47.807770 18299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:47.808017 18294 server_base.cc:1034] running on GCE node
I20260619 16:57:47.808239 18294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:47.808483 18294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:47.809613 18294 hybrid_clock.cc:648] HybridClock initialized: now 1781888267809594 us; error 36 us; skew 500 ppm
Jun 19 16:57:47 dist-test-slave-xb9n krb5kdc[17919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:47.811931 18294 init.cc:377] Logged in from keytab as kudu/127.30.148.131@KRBTEST.COM (short username kudu)
I20260619 16:57:47.812351 18294 webserver.cc:460] Webserver started at http://127.30.148.131:32811/ using document root <none> and password file <none>
I20260619 16:57:47.812525 18294 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:47.812563 18294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:47.812650 18294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:47.813624 18294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "b9b02b3e35774670b4136802e7a3db1c"
format_stamp: "Formatted at 2026-06-19 16:57:47 on dist-test-slave-xb9n"
server_key: "7b43d19397d068abc7dd9bfa355e704e"
server_key_iv: "b01b094e976404297647d28060c52ac8"
server_key_version: "encryptionkey@0"
I20260619 16:57:47.813908 18294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "b9b02b3e35774670b4136802e7a3db1c"
format_stamp: "Formatted at 2026-06-19 16:57:47 on dist-test-slave-xb9n"
server_key: "7b43d19397d068abc7dd9bfa355e704e"
server_key_iv: "b01b094e976404297647d28060c52ac8"
server_key_version: "encryptionkey@0"
I20260619 16:57:47.815153 18294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:57:47.815773 18309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:47.815940 18294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:47.816020 18294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "b9b02b3e35774670b4136802e7a3db1c"
format_stamp: "Formatted at 2026-06-19 16:57:47 on dist-test-slave-xb9n"
server_key: "7b43d19397d068abc7dd9bfa355e704e"
server_key_iv: "b01b094e976404297647d28060c52ac8"
server_key_version: "encryptionkey@0"
I20260619 16:57:47.816071 18294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:47.837575 18294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:47.839665 18294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:47.839768 18294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:47.839920 18294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:47.840204 18294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:47.840238 18294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:47.840258 18294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:47.840287 18294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:47.846022 18294 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:36175
I20260619 16:57:47.846073 18422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:36175 every 8 connection(s)
I20260619 16:57:47.846341 18294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:57:47 dist-test-slave-xb9n krb5kdc[17919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888267, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.131@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:57:47.854687 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 18294
I20260619 16:57:47.854796 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:57:47.854938 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.847048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:39375 (local address 127.30.148.190:38129)
0619 16:57:47.847110 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:47.847112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:47.847293 (+ 181us) server_negotiation.cc:407] Connection header received
0619 16:57:47.847726 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:47.847727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:47.847736 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:47.847788 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:47.848677 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.849175 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.850237 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.850421 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.850483 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.852447 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:47.852449 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:47.852449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:47.852455 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:47.853720 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:47.854069 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:47.854070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:47.854070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:47.854130 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:47.854324 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:47.854325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:47.854325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:47.854416 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:47.854464 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:47.854787 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 16:57:47.854863 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 16:57:47.855033 31314 external_mini_cluster.cc:1428] Setting key 5169fbb9bdfa4281edf7b1d01f745a64
I20260619 16:57:47.855130 18425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.847025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:38129 (local address 127.30.148.131:39375)
0619 16:57:47.847190 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:47.847220 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:47.847639 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:47.847854 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:47.847857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:47.848409 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:47.848585 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.848589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.849254 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.849255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.850092 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.850095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.850495 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.850496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.850589 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.851036 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:57:47.851044 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:47.852365 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:47.853789 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:47.853794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:47.853796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:47.854004 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:47.854185 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:57:47.854186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:57:47.854186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:57:47.854275 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:57:47.854465 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:47.854468 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:47.854631 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 16:57:47.854752 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 16:57:47.854917 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260619 16:57:47.855662 18423 heartbeater.cc:347] Connected to a master server at 127.30.148.190:38129
I20260619 16:57:47.855757 18423 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:47.855928 18423 heartbeater.cc:511] Master 127.30.148.190:38129 requested a full tablet report, sending...
I20260619 16:57:47.856331 17964 ts_manager.cc:194] Registered new tserver with Master: b9b02b3e35774670b4136802e7a3db1c (127.30.148.131:36175)
I20260619 16:57:47.856667 17964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.148.131@KRBTEST.COM'} at 127.30.148.131:39375
I20260619 16:57:47.866658 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:57:47.873116 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.867813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37678 (local address 127.30.148.190:38129)
0619 16:57:47.867848 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:47.867850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:47.867940 (+ 90us) server_negotiation.cc:407] Connection header received
0619 16:57:47.867995 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:47.867995 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:47.868002 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:47.868048 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:47.868566 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.868999 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.869750 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.869921 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.869964 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.870979 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:47.870980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:47.870981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:47.870986 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:47.872309 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:47.872575 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:47.872576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:47.872576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:47.872626 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:47.872761 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:47.872761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:47.872761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:47.872866 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:47.872933 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:47.873008 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 16:57:47.873058 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:57:47.874562 17964 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:57:47.876062 17964 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:57:47.882023 18431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.877879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:44917 (local address 127.0.0.1:44670)
0619 16:57:47.877975 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:47.878010 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:47.878065 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:47.878399 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:47.878403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:47.878405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:47.878462 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.878464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.879500 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.879501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.880815 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.880818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.881731 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.881732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.881815 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.881827 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 16:57:47.881895 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 16:57:47.881941 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":34,"thread_start_us":55,"threads_started":1}
I20260619 16:57:47.882037 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.877985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44670 (local address 127.30.148.130:44917)
0619 16:57:47.878204 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:47.878208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:47.878219 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:57:47.878262 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:47.878264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:47.878280 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:47.878548 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:47.878623 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.879434 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.880914 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.881653 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.881723 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.881771 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:47.881916 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 16:57:47.881959 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":83,"threads_started":1}
I20260619 16:57:47.882117 18433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.878061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:36175 (local address 127.0.0.1:40976)
0619 16:57:47.878305 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:47.878315 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:47.878357 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:47.878489 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:47.878490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:47.878492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:47.878543 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.878545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.879465 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.879466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.880816 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.880818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.881804 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.881805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.881868 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.881887 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 16:57:47.881938 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 16:57:47.881973 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":82,"thread_start_us":67,"threads_started":1}
I20260619 16:57:47.882531 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.878149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40976 (local address 127.30.148.131:36175)
0619 16:57:47.878355 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:47.878360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:47.878369 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:57:47.878407 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:47.878409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:47.878423 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:47.878491 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:47.878599 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.879369 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.880914 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.881652 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.881721 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.881771 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:47.882058 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 16:57:47.882463 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":64,"threads_started":1}
I20260619 16:57:47.882759 18435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.878212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:34703 (local address 127.0.0.1:45158)
0619 16:57:47.878477 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:47.878487 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:47.878529 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:47.879700 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:47.879702 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:47.879704 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:47.879748 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.879749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.880537 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.880538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.881481 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.881484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.882548 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.882549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.882614 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.882618 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:57:47.882660 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 16:57:47.882701 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":45,"threads_started":1}
I20260619 16:57:47.882817 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.878328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45158 (local address 127.30.148.129:34703)
0619 16:57:47.879562 (+ 1234us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:47.879566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:47.879573 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:57:47.879603 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:47.879604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:47.879615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:47.879690 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:47.879803 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.880427 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.881549 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.882448 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.882546 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.882672 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:47.882696 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 16:57:47.882733 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1207,"thread_start_us":70,"threads_started":1}
I20260619 16:57:47.882966 18223 tablet_service.cc:1467] Processing CreateTablet for tablet 1c19f794e0eb41a0a537529e547223ac (DEFAULT_TABLE table=test-table [id=1f472e02ab5b49c8989b355ba380ab4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:47.883119 18357 tablet_service.cc:1467] Processing CreateTablet for tablet 1c19f794e0eb41a0a537529e547223ac (DEFAULT_TABLE table=test-table [id=1f472e02ab5b49c8989b355ba380ab4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:47.883291 18223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c19f794e0eb41a0a537529e547223ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:47.883371 18357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c19f794e0eb41a0a537529e547223ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:47.886019 18089 tablet_service.cc:1467] Processing CreateTablet for tablet 1c19f794e0eb41a0a537529e547223ac (DEFAULT_TABLE table=test-table [id=1f472e02ab5b49c8989b355ba380ab4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:47.886327 18089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c19f794e0eb41a0a537529e547223ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:47.886425 18437 tablet_bootstrap.cc:492] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c: Bootstrap starting.
I20260619 16:57:47.886956 18437 tablet_bootstrap.cc:654] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:47.887414 18437 log.cc:824] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:47.888079 18437 tablet_bootstrap.cc:492] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c: No bootstrap required, opened a new log
I20260619 16:57:47.888204 18437 ts_tablet_manager.cc:1397] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:57:47.888922 18439 tablet_bootstrap.cc:492] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5: Bootstrap starting.
I20260619 16:57:47.889632 18439 tablet_bootstrap.cc:654] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:47.889850 18440 tablet_bootstrap.cc:492] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1: Bootstrap starting.
I20260619 16:57:47.889801 18437 raft_consensus.cc:348] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75c8563fa80428d92d2403e86ff87c1" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 44917 } } peers { permanent_uuid: "b9b02b3e35774670b4136802e7a3db1c" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 36175 } } peers { permanent_uuid: "19714ce9cfe04c9391ae23488ae10fa5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 34703 } }
I20260619 16:57:47.889976 18437 raft_consensus.cc:374] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:47.890017 18437 raft_consensus.cc:729] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9b02b3e35774670b4136802e7a3db1c, State: Initialized, Role: FOLLOWER
I20260619 16:57:47.890184 18439 log.cc:824] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:47.890174 18437 consensus_queue.cc:260] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75c8563fa80428d92d2403e86ff87c1" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 44917 } } peers { permanent_uuid: "b9b02b3e35774670b4136802e7a3db1c" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 36175 } } peers { permanent_uuid: "19714ce9cfe04c9391ae23488ae10fa5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 34703 } }
I20260619 16:57:47.890445 18437 ts_tablet_manager.cc:1428] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:57:47.890504 18440 tablet_bootstrap.cc:654] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:47.890568 18423 heartbeater.cc:503] Master 127.30.148.190:38129 was elected leader, sending a full tablet report...
I20260619 16:57:47.890928 18439 tablet_bootstrap.cc:492] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5: No bootstrap required, opened a new log
I20260619 16:57:47.890996 18439 ts_tablet_manager.cc:1397] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:57:47.891062 18440 log.cc:824] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:47.891742 18440 tablet_bootstrap.cc:492] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1: No bootstrap required, opened a new log
I20260619 16:57:47.891805 18440 ts_tablet_manager.cc:1397] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:57:47.892707 18439 raft_consensus.cc:348] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75c8563fa80428d92d2403e86ff87c1" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 44917 } } peers { permanent_uuid: "b9b02b3e35774670b4136802e7a3db1c" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 36175 } } peers { permanent_uuid: "19714ce9cfe04c9391ae23488ae10fa5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 34703 } }
I20260619 16:57:47.892824 18439 raft_consensus.cc:374] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:47.892844 18439 raft_consensus.cc:729] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19714ce9cfe04c9391ae23488ae10fa5, State: Initialized, Role: FOLLOWER
I20260619 16:57:47.892946 18439 consensus_queue.cc:260] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75c8563fa80428d92d2403e86ff87c1" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 44917 } } peers { permanent_uuid: "b9b02b3e35774670b4136802e7a3db1c" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 36175 } } peers { permanent_uuid: "19714ce9cfe04c9391ae23488ae10fa5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 34703 } }
I20260619 16:57:47.893196 18439 ts_tablet_manager.cc:1428] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:57:47.893253 18155 heartbeater.cc:503] Master 127.30.148.190:38129 was elected leader, sending a full tablet report...
I20260619 16:57:47.893195 18440 raft_consensus.cc:348] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75c8563fa80428d92d2403e86ff87c1" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 44917 } } peers { permanent_uuid: "b9b02b3e35774670b4136802e7a3db1c" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 36175 } } peers { permanent_uuid: "19714ce9cfe04c9391ae23488ae10fa5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 34703 } }
I20260619 16:57:47.893443 18440 raft_consensus.cc:374] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:47.893492 18440 raft_consensus.cc:729] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b75c8563fa80428d92d2403e86ff87c1, State: Initialized, Role: FOLLOWER
I20260619 16:57:47.893604 18440 consensus_queue.cc:260] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75c8563fa80428d92d2403e86ff87c1" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 44917 } } peers { permanent_uuid: "b9b02b3e35774670b4136802e7a3db1c" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 36175 } } peers { permanent_uuid: "19714ce9cfe04c9391ae23488ae10fa5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 34703 } }
I20260619 16:57:47.893867 18440 ts_tablet_manager.cc:1428] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:57:47.893931 18289 heartbeater.cc:503] Master 127.30.148.190:38129 was elected leader, sending a full tablet report...
W20260619 16:57:47.933347 18290 tablet.cc:2369] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:57:47.962347 18444 raft_consensus.cc:482] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:57:47.962488 18444 raft_consensus.cc:504] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75c8563fa80428d92d2403e86ff87c1" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 44917 } } peers { permanent_uuid: "b9b02b3e35774670b4136802e7a3db1c" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 36175 } } peers { permanent_uuid: "19714ce9cfe04c9391ae23488ae10fa5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 34703 } }
I20260619 16:57:47.962841 18444 leader_election.cc:288] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b75c8563fa80428d92d2403e86ff87c1 (127.30.148.130:44917), b9b02b3e35774670b4136802e7a3db1c (127.30.148.131:36175)
I20260619 16:57:47.965708 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.963011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:44917 (local address 127.30.148.129:51227)
0619 16:57:47.963082 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:47.963095 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:47.963153 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:47.963315 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:47.963317 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:47.963318 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:47.963402 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.963404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.964123 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.964124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.964856 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.964858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.965438 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.965439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.965519 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.965522 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:47.965590 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 16:57:47.965623 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 16:57:47.965715 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.963095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:51227 (local address 127.30.148.130:44917)
0619 16:57:47.963149 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:47.963151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:47.963161 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:57:47.963230 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:47.963231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:47.963243 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:47.963294 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:47.963488 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.964040 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.964942 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.965340 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.965443 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.965463 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:47.965607 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 16:57:47.965652 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 16:57:47.965780 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.963196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:60747 (local address 127.30.148.131:36175)
0619 16:57:47.963255 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:47.963258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:47.963377 (+ 119us) server_negotiation.cc:407] Connection header received
0619 16:57:47.963404 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:47.963405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:47.963416 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:47.963473 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:47.963630 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.964136 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.965017 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.965419 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.965486 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.965496 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:47.965670 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 16:57:47.965709 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 16:57:47.965821 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.963142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:36175 (local address 127.30.148.129:60747)
0619 16:57:47.963307 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:47.963315 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:47.963367 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:47.963487 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:47.963488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:47.963490 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:47.963557 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.963559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.964215 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.964216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.964936 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:47.964938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.965495 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.965501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:47.965584 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.965589 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:57:47.965656 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 16:57:47.965692 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":111,"threads_started":1}
I20260619 16:57:47.966065 18243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c19f794e0eb41a0a537529e547223ac" candidate_uuid: "19714ce9cfe04c9391ae23488ae10fa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b75c8563fa80428d92d2403e86ff87c1" is_pre_election: true
I20260619 16:57:47.966204 18243 raft_consensus.cc:2454] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19714ce9cfe04c9391ae23488ae10fa5 in term 0.
I20260619 16:57:47.966284 18377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c19f794e0eb41a0a537529e547223ac" candidate_uuid: "19714ce9cfe04c9391ae23488ae10fa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9b02b3e35774670b4136802e7a3db1c" is_pre_election: true
I20260619 16:57:47.966413 18377 raft_consensus.cc:2454] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19714ce9cfe04c9391ae23488ae10fa5 in term 0.
I20260619 16:57:47.966418 18044 leader_election.cc:302] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19714ce9cfe04c9391ae23488ae10fa5, b75c8563fa80428d92d2403e86ff87c1; no voters:
I20260619 16:57:47.966547 18444 raft_consensus.cc:2781] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:57:47.966593 18444 raft_consensus.cc:482] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:57:47.966616 18444 raft_consensus.cc:3037] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:47.967316 18444 raft_consensus.cc:504] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75c8563fa80428d92d2403e86ff87c1" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 44917 } } peers { permanent_uuid: "b9b02b3e35774670b4136802e7a3db1c" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 36175 } } peers { permanent_uuid: "19714ce9cfe04c9391ae23488ae10fa5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 34703 } }
I20260619 16:57:47.967411 18444 leader_election.cc:288] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [CANDIDATE]: Term 1 election: Requested vote from peers b75c8563fa80428d92d2403e86ff87c1 (127.30.148.130:44917), b9b02b3e35774670b4136802e7a3db1c (127.30.148.131:36175)
I20260619 16:57:47.967615 18243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c19f794e0eb41a0a537529e547223ac" candidate_uuid: "19714ce9cfe04c9391ae23488ae10fa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b75c8563fa80428d92d2403e86ff87c1"
I20260619 16:57:47.967689 18243 raft_consensus.cc:3037] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:47.967675 18377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c19f794e0eb41a0a537529e547223ac" candidate_uuid: "19714ce9cfe04c9391ae23488ae10fa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9b02b3e35774670b4136802e7a3db1c"
I20260619 16:57:47.967794 18377 raft_consensus.cc:3037] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:47.968536 18243 raft_consensus.cc:2454] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19714ce9cfe04c9391ae23488ae10fa5 in term 1.
I20260619 16:57:47.968720 18044 leader_election.cc:302] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19714ce9cfe04c9391ae23488ae10fa5, b75c8563fa80428d92d2403e86ff87c1; no voters:
I20260619 16:57:47.968842 18377 raft_consensus.cc:2454] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19714ce9cfe04c9391ae23488ae10fa5 in term 1.
I20260619 16:57:47.968832 18444 raft_consensus.cc:2781] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:47.968990 18444 raft_consensus.cc:686] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [term 1 LEADER]: Becoming Leader. State: Replica: 19714ce9cfe04c9391ae23488ae10fa5, State: Running, Role: LEADER
I20260619 16:57:47.969094 18444 consensus_queue.cc:237] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75c8563fa80428d92d2403e86ff87c1" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 44917 } } peers { permanent_uuid: "b9b02b3e35774670b4136802e7a3db1c" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 36175 } } peers { permanent_uuid: "19714ce9cfe04c9391ae23488ae10fa5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 34703 } }
I20260619 16:57:47.969867 17964 catalog_manager.cc:5495] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19714ce9cfe04c9391ae23488ae10fa5 (127.30.148.129). New cstate: current_term: 1 leader_uuid: "19714ce9cfe04c9391ae23488ae10fa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b75c8563fa80428d92d2403e86ff87c1" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9b02b3e35774670b4136802e7a3db1c" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 36175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19714ce9cfe04c9391ae23488ae10fa5" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 34703 } health_report { overall_health: HEALTHY } } }
I20260619 16:57:47.995813 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:47.993720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45168 (local address 127.30.148.129:34703)
0619 16:57:47.993788 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:47.993790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:47.993803 (+ 13us) server_negotiation.cc:407] Connection header received
0619 16:57:47.993847 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:47.993848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:47.993863 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:47.993913 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:57:47.994024 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.994465 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.995201 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:47.995381 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:47.995435 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:47.995555 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:57:47.995654 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:57:47.995715 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:47.995745 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 16:57:47.995761 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 16:57:47.997452 18377 raft_consensus.cc:1261] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c [term 1 FOLLOWER]: Refusing update from remote peer 19714ce9cfe04c9391ae23488ae10fa5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:47.997460 18243 raft_consensus.cc:1261] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1 [term 1 FOLLOWER]: Refusing update from remote peer 19714ce9cfe04c9391ae23488ae10fa5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:47.997759 18444 consensus_queue.cc:1035] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9b02b3e35774670b4136802e7a3db1c" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 36175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:47.997932 18447 consensus_queue.cc:1035] T 1c19f794e0eb41a0a537529e547223ac P 19714ce9cfe04c9391ae23488ae10fa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b75c8563fa80428d92d2403e86ff87c1" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:48.001134 18451 mvcc.cc:205] Tried to move back new op lower bound from 7298614345714982912 to 7298614345601990656. Current Snapshot: MvccSnapshot[applied={T|T < 7298614345714982912 or (T in {7298614345714982912})}]
I20260619 16:57:48.002221 17963 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:48.002329 17963 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:48.003041 17963 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 735ec4efe405491ebcaa501f94373a28: Sending DeleteTablet for 3 replicas of tablet 1c19f794e0eb41a0a537529e547223ac
I20260619 16:57:48.003475 18223 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c19f794e0eb41a0a537529e547223ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:47 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:44670
I20260619 16:57:48.003494 18089 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c19f794e0eb41a0a537529e547223ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:47 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:45158
I20260619 16:57:48.003698 18460 tablet_replica.cc:330] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1: stopping tablet replica
I20260619 16:57:48.003775 18357 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c19f794e0eb41a0a537529e547223ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:47 UTC) from {username='kudu', principal='kudu/127.30.148.190@KRBTEST.COM'} at 127.0.0.1:40976
I20260619 16:57:48.003829 18460 raft_consensus.cc:2229] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:48.003855 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 18026
I20260619 16:57:48.004366 18462 tablet_replica.cc:330] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c: stopping tablet replica
I20260619 16:57:48.004463 18462 raft_consensus.cc:2229] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:48.004558 18462 raft_consensus.cc:2258] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:48.004675 18460 raft_consensus.cc:2258] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:48.004978 18462 ts_tablet_manager.cc:1905] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:48.005167 18460 ts_tablet_manager.cc:1905] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:48.006675 18462 ts_tablet_manager.cc:1918] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:48.006726 18460 ts_tablet_manager.cc:1918] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:48.006767 18462 log.cc:1196] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/1c19f794e0eb41a0a537529e547223ac
I20260619 16:57:48.006793 18460 log.cc:1196] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/1c19f794e0eb41a0a537529e547223ac
I20260619 16:57:48.007050 18460 ts_tablet_manager.cc:1939] T 1c19f794e0eb41a0a537529e547223ac P b75c8563fa80428d92d2403e86ff87c1: Deleting consensus metadata
I20260619 16:57:48.007050 18462 ts_tablet_manager.cc:1939] T 1c19f794e0eb41a0a537529e547223ac P b9b02b3e35774670b4136802e7a3db1c: Deleting consensus metadata
I20260619 16:57:48.007419 17950 catalog_manager.cc:4841] TS b9b02b3e35774670b4136802e7a3db1c (127.30.148.131:36175): tablet 1c19f794e0eb41a0a537529e547223ac (table test-table [id=1f472e02ab5b49c8989b355ba380ab4c]) successfully deleted
I20260619 16:57:48.007501 17950 catalog_manager.cc:4841] TS b75c8563fa80428d92d2403e86ff87c1 (127.30.148.130:44917): tablet 1c19f794e0eb41a0a537529e547223ac (table test-table [id=1f472e02ab5b49c8989b355ba380ab4c]) successfully deleted
W20260619 16:57:48.009591 17950 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:34703 (error 108)
I20260619 16:57:48.009850 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 18160
I20260619 16:57:48.010042 18435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.009932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:34703 (local address 127.0.0.1:45174)
0619 16:57:48.009977 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:48.010007 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:34703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 16:57:48.010141 17950 catalog_manager.cc:4584] TS 19714ce9cfe04c9391ae23488ae10fa5 (127.30.148.129:34703): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c19f794e0eb41a0a537529e547223ac: Network error: Client connection negotiation failed: client connection to 127.30.148.129:34703: connect: Connection refused (error 111)
I20260619 16:57:48.014972 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 18294
I20260619 16:57:48.019662 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 17932
2026-06-19T16:57:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1930 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T16:57:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:48Z Disabled control of system clock
I20260619 16:57:48.046393 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:39813
--webserver_interface=127.30.148.190
--webserver_port=0
--builtin_ntp_servers=127.30.148.148:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:39813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 16:57:48.124728 18467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:48.124926 18467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:48.124946 18467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:48.126106 18467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:48.126152 18467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:48.126165 18467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:48.126178 18467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:48.126226 18467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:48.127501 18467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:39813
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:39813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:48.127732 18467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:48.127995 18467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:48.132645 18473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:48.132648 18472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:48.132813 18467 server_base.cc:1034] running on GCE node
W20260619 16:57:48.132755 18475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:48.133133 18467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:48.133390 18467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:48.134534 18467 hybrid_clock.cc:648] HybridClock initialized: now 1781888268134520 us; error 31 us; skew 500 ppm
I20260619 16:57:48.135622 18467 webserver.cc:460] Webserver started at http://127.30.148.190:37069/ using document root <none> and password file <none>
I20260619 16:57:48.135841 18467 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:48.135885 18467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:48.136023 18467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:48.136994 18467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "a6344508b1014bcf9a4e062e0bf99fd1"
format_stamp: "Formatted at 2026-06-19 16:57:48 on dist-test-slave-xb9n"
server_key: "b39dcbda17d902a40ee3f695a310b0f3"
server_key_iv: "096a6cc204dea2f1fdeaa8b164ef2c52"
server_key_version: "encryptionkey@0"
I20260619 16:57:48.137337 18467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "a6344508b1014bcf9a4e062e0bf99fd1"
format_stamp: "Formatted at 2026-06-19 16:57:48 on dist-test-slave-xb9n"
server_key: "b39dcbda17d902a40ee3f695a310b0f3"
server_key_iv: "096a6cc204dea2f1fdeaa8b164ef2c52"
server_key_version: "encryptionkey@0"
I20260619 16:57:48.138625 18467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:48.139319 18481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:48.139487 18467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:48.139569 18467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "a6344508b1014bcf9a4e062e0bf99fd1"
format_stamp: "Formatted at 2026-06-19 16:57:48 on dist-test-slave-xb9n"
server_key: "b39dcbda17d902a40ee3f695a310b0f3"
server_key_iv: "096a6cc204dea2f1fdeaa8b164ef2c52"
server_key_version: "encryptionkey@0"
I20260619 16:57:48.139636 18467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:48.158025 18467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:48.158295 18467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:48.158432 18467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:48.162097 18467 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:39813
I20260619 16:57:48.162130 18533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:39813 every 8 connection(s)
I20260619 16:57:48.162433 18467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:48.163333 18534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:48.165652 18534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1: Bootstrap starting.
I20260619 16:57:48.166189 18534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:48.166467 18534 log.cc:824] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:48.167136 18534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1: No bootstrap required, opened a new log
I20260619 16:57:48.168682 18534 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6344508b1014bcf9a4e062e0bf99fd1" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39813 } }
I20260619 16:57:48.168810 18534 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:48.168855 18534 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6344508b1014bcf9a4e062e0bf99fd1, State: Initialized, Role: FOLLOWER
I20260619 16:57:48.169023 18534 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6344508b1014bcf9a4e062e0bf99fd1" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39813 } }
I20260619 16:57:48.169092 18534 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:48.169114 18534 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:48.169160 18534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:48.169714 18534 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6344508b1014bcf9a4e062e0bf99fd1" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39813 } }
I20260619 16:57:48.169799 18534 leader_election.cc:302] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6344508b1014bcf9a4e062e0bf99fd1; no voters:
I20260619 16:57:48.169972 18534 leader_election.cc:288] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:48.170020 18537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:48.170226 18537 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [term 1 LEADER]: Becoming Leader. State: Replica: a6344508b1014bcf9a4e062e0bf99fd1, State: Running, Role: LEADER
I20260619 16:57:48.170297 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 18467
I20260619 16:57:48.170229 18534 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:48.170404 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:48.170389 18537 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6344508b1014bcf9a4e062e0bf99fd1" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39813 } }
I20260619 16:57:48.170617 31314 external_mini_cluster.cc:1428] Setting key 99b7e1f03df3288e24c9dcbf893a9ad9
I20260619 16:57:48.170939 18538 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6344508b1014bcf9a4e062e0bf99fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6344508b1014bcf9a4e062e0bf99fd1" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39813 } } }
I20260619 16:57:48.171022 18539 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6344508b1014bcf9a4e062e0bf99fd1. Latest consensus state: current_term: 1 leader_uuid: "a6344508b1014bcf9a4e062e0bf99fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6344508b1014bcf9a4e062e0bf99fd1" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 39813 } } }
I20260619 16:57:48.171023 18538 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:48.171129 18539 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:48.171303 18546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:48.171789 18546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:48.173262 18546 catalog_manager.cc:1347] Generated new cluster ID: 7340a64aa1074192a4179e9c67c9c90d
I20260619 16:57:48.173314 18546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:48.177954 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.171078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.30.148.190:39813)
0619 16:57:48.171904 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:48.171913 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:48.172140 (+ 227us) server_negotiation.cc:407] Connection header received
0619 16:57:48.172440 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:48.172447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:48.172515 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:48.172662 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:48.173000 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.174354 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.175476 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.176099 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.176214 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.177120 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:48.177124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:48.177124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:48.177319 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:48.177389 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:48.177404 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:48.177462 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:48.177667 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 16:57:48.177743 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":80,"threads_started":1}
I20260619 16:57:48.191181 18546 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:48.191879 18546 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:48.196180 18546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1: Generated new TSK 0
I20260619 16:57:48.196345 18546 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:57:48.202541 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.129:0
--local_ip_for_outbound_sockets=127.30.148.129
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39813
--builtin_ntp_servers=127.30.148.148:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 16:57:48.281136 18558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:48.281322 18558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:48.281343 18558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:48.282464 18558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:48.282502 18558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:48.282526 18558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.129
I20260619 16:57:48.283818 18558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.148.129
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.129
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:48.284042 18558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:48.284329 18558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:48.289042 18564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:48.289022 18563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:48.289172 18558 server_base.cc:1034] running on GCE node
W20260619 16:57:48.289212 18566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:48.289490 18558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:48.289739 18558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:48.290889 18558 hybrid_clock.cc:648] HybridClock initialized: now 1781888268290877 us; error 29 us; skew 500 ppm
I20260619 16:57:48.292052 18558 webserver.cc:460] Webserver started at http://127.30.148.129:36977/ using document root <none> and password file <none>
I20260619 16:57:48.292287 18558 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:48.292351 18558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:48.292718 18558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:48.293793 18558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/data/instance:
uuid: "0b0d2fe30b0a452c81bf0fc608c2156e"
format_stamp: "Formatted at 2026-06-19 16:57:48 on dist-test-slave-xb9n"
server_key: "81dc17049e22440840e3dd5ef7a2061c"
server_key_iv: "c62f621fb3b6d2053164570f49da4a93"
server_key_version: "encryptionkey@0"
I20260619 16:57:48.294142 18558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance:
uuid: "0b0d2fe30b0a452c81bf0fc608c2156e"
format_stamp: "Formatted at 2026-06-19 16:57:48 on dist-test-slave-xb9n"
server_key: "81dc17049e22440840e3dd5ef7a2061c"
server_key_iv: "c62f621fb3b6d2053164570f49da4a93"
server_key_version: "encryptionkey@0"
I20260619 16:57:48.295424 18558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:57:48.296123 18572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:48.296291 18558 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:48.296391 18558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/wal
uuid: "0b0d2fe30b0a452c81bf0fc608c2156e"
format_stamp: "Formatted at 2026-06-19 16:57:48 on dist-test-slave-xb9n"
server_key: "81dc17049e22440840e3dd5ef7a2061c"
server_key_iv: "c62f621fb3b6d2053164570f49da4a93"
server_key_version: "encryptionkey@0"
I20260619 16:57:48.296478 18558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:48.318998 18558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:48.319280 18558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:48.319414 18558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:48.319608 18558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:48.319890 18558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:48.319937 18558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:48.319976 18558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:48.320001 18558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:48.325407 18558 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.129:36543
I20260619 16:57:48.325464 18685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.129:36543 every 8 connection(s)
I20260619 16:57:48.325744 18558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/data/info.pb
I20260619 16:57:48.326962 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 18558
I20260619 16:57:48.327076 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-0/wal/instance
I20260619 16:57:48.327282 31314 external_mini_cluster.cc:1428] Setting key abf63d2eb4086e226ac9f774dd882c36
I20260619 16:57:48.328295 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.130:0
--local_ip_for_outbound_sockets=127.30.148.130
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39813
--builtin_ntp_servers=127.30.148.148:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 16:57:48.331205 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.326347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.129:37205 (local address 127.30.148.190:39813)
0619 16:57:48.326416 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:48.326419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:48.326640 (+ 221us) server_negotiation.cc:407] Connection header received
0619 16:57:48.327094 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:48.327095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:48.327108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:48.327165 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:48.327975 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.328729 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.329849 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.330039 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.330112 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.330918 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:48.330919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:48.330919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:48.330923 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:48.330936 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:48.330941 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:48.331008 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:48.331096 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 16:57:48.331123 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 16:57:48.331405 18688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.326338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:39813 (local address 127.30.148.129:37205)
0619 16:57:48.326530 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:48.326562 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:48.326992 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:48.327228 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:48.327232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:48.327241 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:48.327869 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.327875 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.328813 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.328813 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.329751 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.329753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.330118 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.330119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.330256 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.330779 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:57:48.330779 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:48.330805 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:57:48.330825 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:48.331008 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:48.331014 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:48.331017 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:48.331101 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 16:57:48.331196 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260619 16:57:48.331905 18686 heartbeater.cc:347] Connected to a master server at 127.30.148.190:39813
I20260619 16:57:48.332010 18686 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:48.332211 18686 heartbeater.cc:511] Master 127.30.148.190:39813 requested a full tablet report, sending...
I20260619 16:57:48.332723 18498 ts_manager.cc:194] Registered new tserver with Master: 0b0d2fe30b0a452c81bf0fc608c2156e (127.30.148.129:36543)
I20260619 16:57:48.333307 18498 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.148.129:37205
W20260619 16:57:48.409989 18689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:48.410173 18689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:48.410194 18689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:48.411295 18689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:48.411332 18689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:48.411381 18689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.130
I20260619 16:57:48.412689 18689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.148.130
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.130
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:48.412914 18689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:48.413203 18689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:48.417820 18697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:48.417853 18694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:48.417834 18695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:48.418076 18689 server_base.cc:1034] running on GCE node
I20260619 16:57:48.418269 18689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:48.418494 18689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:48.419656 18689 hybrid_clock.cc:648] HybridClock initialized: now 1781888268419643 us; error 29 us; skew 500 ppm
I20260619 16:57:48.420900 18689 webserver.cc:460] Webserver started at http://127.30.148.130:38637/ using document root <none> and password file <none>
I20260619 16:57:48.421154 18689 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:48.421221 18689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:48.421393 18689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:48.422380 18689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/data/instance:
uuid: "af8f17661a4e4724ac89625bd319e49f"
format_stamp: "Formatted at 2026-06-19 16:57:48 on dist-test-slave-xb9n"
server_key: "a97d6e5421dd6392a66adef2eed49c7a"
server_key_iv: "c028c8fc0432296a335beea81c818271"
server_key_version: "encryptionkey@0"
I20260619 16:57:48.422722 18689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance:
uuid: "af8f17661a4e4724ac89625bd319e49f"
format_stamp: "Formatted at 2026-06-19 16:57:48 on dist-test-slave-xb9n"
server_key: "a97d6e5421dd6392a66adef2eed49c7a"
server_key_iv: "c028c8fc0432296a335beea81c818271"
server_key_version: "encryptionkey@0"
I20260619 16:57:48.423929 18689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:57:48.424585 18703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:48.424744 18689 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:57:48.424834 18689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/wal
uuid: "af8f17661a4e4724ac89625bd319e49f"
format_stamp: "Formatted at 2026-06-19 16:57:48 on dist-test-slave-xb9n"
server_key: "a97d6e5421dd6392a66adef2eed49c7a"
server_key_iv: "c028c8fc0432296a335beea81c818271"
server_key_version: "encryptionkey@0"
I20260619 16:57:48.424906 18689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:48.433733 18689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:48.433964 18689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:48.434089 18689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:48.434284 18689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:48.434585 18689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:48.434633 18689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:48.434671 18689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:48.434697 18689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:48.440338 18689 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.130:45465
I20260619 16:57:48.440375 18816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.130:45465 every 8 connection(s)
I20260619 16:57:48.440677 18689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/data/info.pb
I20260619 16:57:48.442864 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 18689
I20260619 16:57:48.442996 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/wal/instance
I20260619 16:57:48.443212 31314 external_mini_cluster.cc:1428] Setting key 8357447e0bf749b88c40f4d8c4feb650
I20260619 16:57:48.444042 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.148.131:0
--local_ip_for_outbound_sockets=127.30.148.131
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39813
--builtin_ntp_servers=127.30.148.148:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 16:57:48.445914 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.441345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.130:51981 (local address 127.30.148.190:39813)
0619 16:57:48.441416 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:48.441418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:48.441625 (+ 207us) server_negotiation.cc:407] Connection header received
0619 16:57:48.442154 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:48.442155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:48.442167 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:48.442228 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:48.442836 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.443545 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.444540 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.444741 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.444810 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.445644 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:48.445646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:48.445647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:48.445651 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:48.445664 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:48.445668 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:48.445724 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:48.445811 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 16:57:48.445839 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 16:57:48.446102 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.441336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:39813 (local address 127.30.148.130:51981)
0619 16:57:48.441513 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:48.441543 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:48.442051 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:48.442291 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:48.442295 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:48.442304 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:48.442750 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.442754 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.443624 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.443625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.444452 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.444454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.444815 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.444816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.444935 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.445488 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:57:48.445489 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:48.445515 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:57:48.445539 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:48.445725 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:48.445730 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:48.445733 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:48.445813 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 16:57:48.445908 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260619 16:57:48.446535 18817 heartbeater.cc:347] Connected to a master server at 127.30.148.190:39813
I20260619 16:57:48.446632 18817 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:48.446813 18817 heartbeater.cc:511] Master 127.30.148.190:39813 requested a full tablet report, sending...
I20260619 16:57:48.447150 18498 ts_manager.cc:194] Registered new tserver with Master: af8f17661a4e4724ac89625bd319e49f (127.30.148.130:45465)
I20260619 16:57:48.447507 18498 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.148.130:51981
W20260619 16:57:48.523952 18820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:48.524150 18820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:48.524173 18820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:48.525285 18820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:48.525324 18820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:48.525393 18820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.148.131
I20260619 16:57:48.526723 18820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.148.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.148.131
--webserver_port=0
--tserver_master_addrs=127.30.148.190:39813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.148.131
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:48.526942 18820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:48.527256 18820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:48.532193 18826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:48.532195 18825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:48.532195 18828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:48.532369 18820 server_base.cc:1034] running on GCE node
I20260619 16:57:48.532575 18820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:48.532805 18820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:48.533979 18820 hybrid_clock.cc:648] HybridClock initialized: now 1781888268533948 us; error 38 us; skew 500 ppm
I20260619 16:57:48.535210 18820 webserver.cc:460] Webserver started at http://127.30.148.131:44119/ using document root <none> and password file <none>
I20260619 16:57:48.535432 18820 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:48.535497 18820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:48.535645 18820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:48.536633 18820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/data/instance:
uuid: "832a19a51825421db3d99f5bf4fee2ff"
format_stamp: "Formatted at 2026-06-19 16:57:48 on dist-test-slave-xb9n"
server_key: "010736e5d273b2aef6841e8fa8df42b8"
server_key_iv: "eb81fdb82b9ab3cbbbc1cfd42c3e9857"
server_key_version: "encryptionkey@0"
I20260619 16:57:48.536973 18820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance:
uuid: "832a19a51825421db3d99f5bf4fee2ff"
format_stamp: "Formatted at 2026-06-19 16:57:48 on dist-test-slave-xb9n"
server_key: "010736e5d273b2aef6841e8fa8df42b8"
server_key_iv: "eb81fdb82b9ab3cbbbc1cfd42c3e9857"
server_key_version: "encryptionkey@0"
I20260619 16:57:48.538267 18820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:57:48.538990 18834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:48.539155 18820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:57:48.539244 18820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/wal
uuid: "832a19a51825421db3d99f5bf4fee2ff"
format_stamp: "Formatted at 2026-06-19 16:57:48 on dist-test-slave-xb9n"
server_key: "010736e5d273b2aef6841e8fa8df42b8"
server_key_iv: "eb81fdb82b9ab3cbbbc1cfd42c3e9857"
server_key_version: "encryptionkey@0"
I20260619 16:57:48.539315 18820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:48.556581 18820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:48.556861 18820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:48.557006 18820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:48.557229 18820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:57:48.557554 18820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:57:48.557603 18820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:48.557637 18820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:57:48.557663 18820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:48.562917 18820 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.131:39555
I20260619 16:57:48.563002 18947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.131:39555 every 8 connection(s)
I20260619 16:57:48.563293 18820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/data/info.pb
I20260619 16:57:48.568270 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.563963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:46951 (local address 127.30.148.190:39813)
0619 16:57:48.564025 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:48.564027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:48.564264 (+ 237us) server_negotiation.cc:407] Connection header received
0619 16:57:48.564736 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:48.564737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:48.564748 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:48.564811 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:48.565439 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.565958 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.566909 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.567117 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.567185 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.567976 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:48.567978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:48.567978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:48.567982 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:48.567996 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:48.568000 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:48.568044 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:48.568153 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 16:57:48.568190 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 16:57:48.568488 18950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.563968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:39813 (local address 127.30.148.131:46951)
0619 16:57:48.564146 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:48.564182 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:48.564640 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:48.564876 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:48.564879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:48.564886 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:57:48.565336 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.565340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.566066 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.566067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.566824 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.566826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.567205 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.567206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.567335 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.567841 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:57:48.567842 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:57:48.567867 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:57:48.567886 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:57:48.568043 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:57:48.568047 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:57:48.568050 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:48.568154 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 16:57:48.568280 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260619 16:57:48.568898 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 18820
I20260619 16:57:48.568953 18948 heartbeater.cc:347] Connected to a master server at 127.30.148.190:39813
I20260619 16:57:48.569022 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/wal/instance
I20260619 16:57:48.569056 18948 heartbeater.cc:464] Registering TS with master...
I20260619 16:57:48.569248 18948 heartbeater.cc:511] Master 127.30.148.190:39813 requested a full tablet report, sending...
I20260619 16:57:48.569255 31314 external_mini_cluster.cc:1428] Setting key 2b2d1ccff8599884dcae34a582f56892
I20260619 16:57:48.569639 18498 ts_manager.cc:194] Registered new tserver with Master: 832a19a51825421db3d99f5bf4fee2ff (127.30.148.131:39555)
I20260619 16:57:48.570163 18498 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.148.131:46951
I20260619 16:57:48.570192 31314 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:57:48.574136 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.571708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36738 (local address 127.30.148.190:39813)
0619 16:57:48.571744 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:48.571746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:48.571885 (+ 139us) server_negotiation.cc:407] Connection header received
0619 16:57:48.571919 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:48.571919 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:48.571929 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:48.571971 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:48.572102 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.572521 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.573204 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.573418 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.573476 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.573954 (+ 478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:48.573955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:48.573956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:48.573959 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:48.573969 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:48.573972 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:48.574012 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:48.574043 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 16:57:48.574060 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 16:57:48.575485 18498 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:57:48.576910 18498 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:57:48.582854 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.579041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.30.148.131:39555)
0619 16:57:48.579284 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:48.579287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:48.579329 (+ 42us) server_negotiation.cc:407] Connection header received
0619 16:57:48.579402 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:48.579403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:48.579415 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:48.579503 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:48.579629 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.580732 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.581812 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.582469 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.582544 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.582622 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:48.582745 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 16:57:48.582780 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":98,"threads_started":1}
I20260619 16:57:48.582855 18956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.578889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.131:39555 (local address 127.0.0.1:53276)
0619 16:57:48.579281 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:48.579290 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:48.579337 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:48.579503 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:48.579504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:48.579506 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:48.579550 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.579552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.580830 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.580831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.581728 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.581730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.582562 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.582563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.582642 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.582655 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 16:57:48.582728 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 16:57:48.582779 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":1,"spinlock_wait_cycles":768,"thread_start_us":86,"threads_started":1}
I20260619 16:57:48.583045 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.579065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37394 (local address 127.30.148.130:45465)
0619 16:57:48.579248 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:48.579253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:48.579264 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:57:48.579308 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:48.579310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:48.579325 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:48.579398 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:48.579633 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.580731 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.582021 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.582487 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.582574 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.582656 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:48.582895 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 16:57:48.582941 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260619 16:57:48.583104 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.578899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36543 (local address 127.0.0.1:48138)
0619 16:57:48.579425 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:48.579433 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:48.579474 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:48.579724 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:48.579725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:48.579727 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:48.579763 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.579764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.580627 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.580628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.581911 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.581914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.582874 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.582875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.582947 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.582952 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:57:48.583009 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 16:57:48.583044 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260619 16:57:48.583200 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.578889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:45465 (local address 127.0.0.1:37394)
0619 16:57:48.579062 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:48.579096 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:48.579182 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:48.579469 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:48.579472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:48.579474 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:48.579530 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.579532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.580825 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.580825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.581936 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.581938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.582688 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.582689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.582748 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.582751 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:48.582795 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 16:57:48.582825 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":24,"thread_start_us":71,"threads_started":1}
I20260619 16:57:48.583374 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.579121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48138 (local address 127.30.148.129:36543)
0619 16:57:48.579255 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:48.579259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:48.579532 (+ 273us) server_negotiation.cc:407] Connection header received
0619 16:57:48.579575 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:48.579577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:48.579592 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:48.579659 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:48.579838 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.580529 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.581998 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.582699 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.582764 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.582838 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:48.583247 (+ 409us) server_negotiation.cc:299] Negotiation successful
0619 16:57:48.583288 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260619 16:57:48.583905 18620 tablet_service.cc:1467] Processing CreateTablet for tablet 31adfb0eecf04a46ba1dee47d3ea5ada (DEFAULT_TABLE table=test-table [id=ead5dae7d0d741a7937a0f7931cb8f91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:48.583902 18882 tablet_service.cc:1467] Processing CreateTablet for tablet 31adfb0eecf04a46ba1dee47d3ea5ada (DEFAULT_TABLE table=test-table [id=ead5dae7d0d741a7937a0f7931cb8f91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:48.583901 18751 tablet_service.cc:1467] Processing CreateTablet for tablet 31adfb0eecf04a46ba1dee47d3ea5ada (DEFAULT_TABLE table=test-table [id=ead5dae7d0d741a7937a0f7931cb8f91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:57:48.584242 18751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31adfb0eecf04a46ba1dee47d3ea5ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:48.584254 18620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31adfb0eecf04a46ba1dee47d3ea5ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:48.584245 18882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31adfb0eecf04a46ba1dee47d3ea5ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:48.586833 18962 tablet_bootstrap.cc:492] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f: Bootstrap starting.
I20260619 16:57:48.587379 18964 tablet_bootstrap.cc:492] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff: Bootstrap starting.
I20260619 16:57:48.587625 18962 tablet_bootstrap.cc:654] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:48.587942 18962 log.cc:824] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:48.587984 18964 tablet_bootstrap.cc:654] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:48.588289 18964 log.cc:824] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:48.588582 18963 tablet_bootstrap.cc:492] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e: Bootstrap starting.
I20260619 16:57:48.588747 18962 tablet_bootstrap.cc:492] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f: No bootstrap required, opened a new log
I20260619 16:57:48.588816 18962 ts_tablet_manager.cc:1397] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:57:48.588974 18964 tablet_bootstrap.cc:492] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff: No bootstrap required, opened a new log
I20260619 16:57:48.589046 18964 ts_tablet_manager.cc:1397] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:57:48.589208 18963 tablet_bootstrap.cc:654] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:48.589591 18963 log.cc:824] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:48.590484 18963 tablet_bootstrap.cc:492] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e: No bootstrap required, opened a new log
I20260619 16:57:48.590548 18963 ts_tablet_manager.cc:1397] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 16:57:48.591241 18964 raft_consensus.cc:348] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832a19a51825421db3d99f5bf4fee2ff" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39555 } } peers { permanent_uuid: "af8f17661a4e4724ac89625bd319e49f" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45465 } } peers { permanent_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36543 } }
I20260619 16:57:48.591380 18964 raft_consensus.cc:374] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:48.591421 18964 raft_consensus.cc:729] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 832a19a51825421db3d99f5bf4fee2ff, State: Initialized, Role: FOLLOWER
I20260619 16:57:48.591536 18964 consensus_queue.cc:260] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832a19a51825421db3d99f5bf4fee2ff" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39555 } } peers { permanent_uuid: "af8f17661a4e4724ac89625bd319e49f" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45465 } } peers { permanent_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36543 } }
I20260619 16:57:48.591491 18962 raft_consensus.cc:348] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832a19a51825421db3d99f5bf4fee2ff" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39555 } } peers { permanent_uuid: "af8f17661a4e4724ac89625bd319e49f" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45465 } } peers { permanent_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36543 } }
I20260619 16:57:48.591640 18962 raft_consensus.cc:374] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:48.591684 18962 raft_consensus.cc:729] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af8f17661a4e4724ac89625bd319e49f, State: Initialized, Role: FOLLOWER
I20260619 16:57:48.591795 18964 ts_tablet_manager.cc:1428] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:57:48.591799 18962 consensus_queue.cc:260] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832a19a51825421db3d99f5bf4fee2ff" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39555 } } peers { permanent_uuid: "af8f17661a4e4724ac89625bd319e49f" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45465 } } peers { permanent_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36543 } }
I20260619 16:57:48.591900 18948 heartbeater.cc:503] Master 127.30.148.190:39813 was elected leader, sending a full tablet report...
I20260619 16:57:48.592303 18817 heartbeater.cc:503] Master 127.30.148.190:39813 was elected leader, sending a full tablet report...
I20260619 16:57:48.592483 18962 ts_tablet_manager.cc:1428] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260619 16:57:48.592887 18963 raft_consensus.cc:348] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832a19a51825421db3d99f5bf4fee2ff" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39555 } } peers { permanent_uuid: "af8f17661a4e4724ac89625bd319e49f" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45465 } } peers { permanent_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36543 } }
I20260619 16:57:48.593032 18963 raft_consensus.cc:374] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:48.593076 18963 raft_consensus.cc:729] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0d2fe30b0a452c81bf0fc608c2156e, State: Initialized, Role: FOLLOWER
I20260619 16:57:48.593192 18963 consensus_queue.cc:260] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832a19a51825421db3d99f5bf4fee2ff" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39555 } } peers { permanent_uuid: "af8f17661a4e4724ac89625bd319e49f" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45465 } } peers { permanent_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36543 } }
I20260619 16:57:48.593465 18963 ts_tablet_manager.cc:1428] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 16:57:48.593516 18686 heartbeater.cc:503] Master 127.30.148.190:39813 was elected leader, sending a full tablet report...
W20260619 16:57:48.691207 18818 tablet.cc:2369] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:57:48.715541 18968 raft_consensus.cc:482] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:57:48.715698 18968 raft_consensus.cc:504] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832a19a51825421db3d99f5bf4fee2ff" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39555 } } peers { permanent_uuid: "af8f17661a4e4724ac89625bd319e49f" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45465 } } peers { permanent_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36543 } }
I20260619 16:57:48.716071 18968 leader_election.cc:288] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af8f17661a4e4724ac89625bd319e49f (127.30.148.130:45465), 0b0d2fe30b0a452c81bf0fc608c2156e (127.30.148.129:36543)
I20260619 16:57:48.718868 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.716323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:52237 (local address 127.30.148.130:45465)
0619 16:57:48.716368 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:48.716370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:48.716383 (+ 13us) server_negotiation.cc:407] Connection header received
0619 16:57:48.716425 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:48.716426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:48.716435 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:48.716498 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:48.716657 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.717174 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.718190 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.718536 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.718611 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.718620 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:48.718762 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 16:57:48.718797 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:57:48.718900 18950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.716244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.130:45465 (local address 127.30.148.131:52237)
0619 16:57:48.716299 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:48.716312 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:48.716358 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:48.716500 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:48.716501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:48.716502 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:48.716573 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.716574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.717318 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.717319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.718099 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.718102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.718619 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.718620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.718686 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.718689 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:48.718756 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 16:57:48.718796 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 16:57:48.718904 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.716352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.148.131:33959 (local address 127.30.148.129:36543)
0619 16:57:48.716400 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:48.716402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:48.716610 (+ 208us) server_negotiation.cc:407] Connection header received
0619 16:57:48.716671 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:48.716672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:48.716684 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:48.716735 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:57:48.716887 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.717323 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.718234 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.718578 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.718661 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.718668 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:48.718804 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 16:57:48.718838 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:57:48.719126 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.716259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36543 (local address 127.30.148.131:33959)
0619 16:57:48.716537 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:48.716545 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:57:48.716615 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:57:48.716741 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:57:48.716741 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:57:48.716743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:57:48.716809 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.716810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.717442 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.717443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.718149 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:57:48.718150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.718659 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.718660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:57:48.718730 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.718733 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:57:48.718787 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 16:57:48.718824 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":119,"thread_start_us":102,"threads_started":1}
I20260619 16:57:48.719278 18771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31adfb0eecf04a46ba1dee47d3ea5ada" candidate_uuid: "832a19a51825421db3d99f5bf4fee2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af8f17661a4e4724ac89625bd319e49f" is_pre_election: true
I20260619 16:57:48.719439 18771 raft_consensus.cc:2454] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 832a19a51825421db3d99f5bf4fee2ff in term 0.
I20260619 16:57:48.719451 18640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31adfb0eecf04a46ba1dee47d3ea5ada" candidate_uuid: "832a19a51825421db3d99f5bf4fee2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e" is_pre_election: true
I20260619 16:57:48.719573 18640 raft_consensus.cc:2454] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 832a19a51825421db3d99f5bf4fee2ff in term 0.
I20260619 16:57:48.719679 18836 leader_election.cc:302] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 832a19a51825421db3d99f5bf4fee2ff, af8f17661a4e4724ac89625bd319e49f; no voters:
I20260619 16:57:48.719794 18968 raft_consensus.cc:2781] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:57:48.719844 18968 raft_consensus.cc:482] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:57:48.719866 18968 raft_consensus.cc:3037] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:48.720566 18968 raft_consensus.cc:504] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832a19a51825421db3d99f5bf4fee2ff" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39555 } } peers { permanent_uuid: "af8f17661a4e4724ac89625bd319e49f" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45465 } } peers { permanent_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36543 } }
I20260619 16:57:48.720714 18968 leader_election.cc:288] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [CANDIDATE]: Term 1 election: Requested vote from peers af8f17661a4e4724ac89625bd319e49f (127.30.148.130:45465), 0b0d2fe30b0a452c81bf0fc608c2156e (127.30.148.129:36543)
I20260619 16:57:48.720904 18640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31adfb0eecf04a46ba1dee47d3ea5ada" candidate_uuid: "832a19a51825421db3d99f5bf4fee2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e"
I20260619 16:57:48.720904 18771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31adfb0eecf04a46ba1dee47d3ea5ada" candidate_uuid: "832a19a51825421db3d99f5bf4fee2ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af8f17661a4e4724ac89625bd319e49f"
I20260619 16:57:48.720981 18771 raft_consensus.cc:3037] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:48.720981 18640 raft_consensus.cc:3037] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:48.721897 18640 raft_consensus.cc:2454] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 832a19a51825421db3d99f5bf4fee2ff in term 1.
I20260619 16:57:48.721897 18771 raft_consensus.cc:2454] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 832a19a51825421db3d99f5bf4fee2ff in term 1.
I20260619 16:57:48.722048 18836 leader_election.cc:302] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 832a19a51825421db3d99f5bf4fee2ff, af8f17661a4e4724ac89625bd319e49f; no voters:
I20260619 16:57:48.722167 18968 raft_consensus.cc:2781] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:48.722313 18968 raft_consensus.cc:686] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 1 LEADER]: Becoming Leader. State: Replica: 832a19a51825421db3d99f5bf4fee2ff, State: Running, Role: LEADER
I20260619 16:57:48.722426 18968 consensus_queue.cc:237] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832a19a51825421db3d99f5bf4fee2ff" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39555 } } peers { permanent_uuid: "af8f17661a4e4724ac89625bd319e49f" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45465 } } peers { permanent_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36543 } }
I20260619 16:57:48.723171 18497 catalog_manager.cc:5495] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 832a19a51825421db3d99f5bf4fee2ff (127.30.148.131). New cstate: current_term: 1 leader_uuid: "832a19a51825421db3d99f5bf4fee2ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "832a19a51825421db3d99f5bf4fee2ff" member_type: VOTER last_known_addr { host: "127.30.148.131" port: 39555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af8f17661a4e4724ac89625bd319e49f" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36543 } health_report { overall_health: UNKNOWN } } }
I20260619 16:57:48.761119 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.759132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53288 (local address 127.30.148.131:39555)
0619 16:57:48.759180 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:48.759182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:48.759191 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:57:48.759220 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:48.759220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:48.759234 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:48.759298 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:57:48.759408 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.759856 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.760535 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:48.760706 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:48.760749 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:48.760866 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:57:48.760950 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:57:48.761007 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:48.761037 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 16:57:48.761050 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:57:48.762792 18640 raft_consensus.cc:1261] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e [term 1 FOLLOWER]: Refusing update from remote peer 832a19a51825421db3d99f5bf4fee2ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:48.762792 18771 raft_consensus.cc:1261] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f [term 1 FOLLOWER]: Refusing update from remote peer 832a19a51825421db3d99f5bf4fee2ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:57:48.763114 18968 consensus_queue.cc:1035] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0d2fe30b0a452c81bf0fc608c2156e" member_type: VOTER last_known_addr { host: "127.30.148.129" port: 36543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:48.763260 18972 consensus_queue.cc:1035] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "af8f17661a4e4724ac89625bd319e49f" member_type: VOTER last_known_addr { host: "127.30.148.130" port: 45465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:57:48.767645 18498 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:48.767732 18498 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:57:48.768396 18498 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6344508b1014bcf9a4e062e0bf99fd1: Sending DeleteTablet for 3 replicas of tablet 31adfb0eecf04a46ba1dee47d3ea5ada
I20260619 16:57:48.768687 18751 tablet_service.cc:1514] Processing DeleteTablet for tablet 31adfb0eecf04a46ba1dee47d3ea5ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:48 UTC) from {username='slave'} at 127.0.0.1:37394
I20260619 16:57:48.768715 18620 tablet_service.cc:1514] Processing DeleteTablet for tablet 31adfb0eecf04a46ba1dee47d3ea5ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:48 UTC) from {username='slave'} at 127.0.0.1:48138
I20260619 16:57:48.768924 18986 tablet_replica.cc:330] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e: stopping tablet replica
I20260619 16:57:48.768922 18882 tablet_service.cc:1514] Processing DeleteTablet for tablet 31adfb0eecf04a46ba1dee47d3ea5ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:57:48 UTC) from {username='slave'} at 127.0.0.1:53276
I20260619 16:57:48.769016 18986 raft_consensus.cc:2229] T 31adfb0eecf04a46ba1dee47d3ea5ada P 0b0d2fe30b0a452c81bf0fc608c2156e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:48.769054 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 18558
I20260619 16:57:48.769672 18987 tablet_replica.cc:330] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff: stopping tablet replica
I20260619 16:57:48.769753 18987 raft_consensus.cc:2229] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:57:48.769834 18987 raft_consensus.cc:2258] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:48.770165 18987 ts_tablet_manager.cc:1905] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:48.771665 18987 ts_tablet_manager.cc:1918] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:48.771711 18987 log.cc:1196] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-2/wal/wals/31adfb0eecf04a46ba1dee47d3ea5ada
I20260619 16:57:48.771885 18987 ts_tablet_manager.cc:1939] T 31adfb0eecf04a46ba1dee47d3ea5ada P 832a19a51825421db3d99f5bf4fee2ff: Deleting consensus metadata
I20260619 16:57:48.772250 18485 catalog_manager.cc:4841] TS 832a19a51825421db3d99f5bf4fee2ff (127.30.148.131:39555): tablet 31adfb0eecf04a46ba1dee47d3ea5ada (table test-table [id=ead5dae7d0d741a7937a0f7931cb8f91]) successfully deleted
I20260619 16:57:48.773455 18985 tablet_replica.cc:330] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f: stopping tablet replica
I20260619 16:57:48.773552 18985 raft_consensus.cc:2229] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:57:48.773643 18985 raft_consensus.cc:2258] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:57:48.773998 18985 ts_tablet_manager.cc:1905] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:57:48.775195 18985 ts_tablet_manager.cc:1918] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:57:48.775269 18985 log.cc:1196] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f: Deleting WAL directory at /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888036930978-31314-0/minicluster-data/ts-1/wal/wals/31adfb0eecf04a46ba1dee47d3ea5ada
I20260619 16:57:48.775509 18985 ts_tablet_manager.cc:1939] T 31adfb0eecf04a46ba1dee47d3ea5ada P af8f17661a4e4724ac89625bd319e49f: Deleting consensus metadata
I20260619 16:57:48.775853 18483 catalog_manager.cc:4841] TS af8f17661a4e4724ac89625bd319e49f (127.30.148.130:45465): tablet 31adfb0eecf04a46ba1dee47d3ea5ada (table test-table [id=ead5dae7d0d741a7937a0f7931cb8f91]) successfully deleted
W20260619 16:57:48.776665 18482 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.148.129:36543 (error 108)
I20260619 16:57:48.777100 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:48.776979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.129:36543 (local address 127.0.0.1:48150)
0619 16:57:48.777026 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 16:57:48.777063 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.129:36543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 16:57:48.777181 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 18689
W20260619 16:57:48.779565 18482 catalog_manager.cc:4584] TS 0b0d2fe30b0a452c81bf0fc608c2156e (127.30.148.129:36543): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31adfb0eecf04a46ba1dee47d3ea5ada: Network error: Client connection negotiation failed: client connection to 127.30.148.129:36543: connect: Connection refused (error 111)
I20260619 16:57:48.782610 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 18820
I20260619 16:57:48.786832 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 18467
2026-06-19T16:57:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (765 ms)
[----------] 6 tests from EncryptionPolicyTest (8179 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:57:48 dist-test-slave-xb9n krb5kdc[18991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:57:48 dist-test-slave-xb9n krb5kdc[18991](info): set up 2 sockets
Jun 19 16:57:48 dist-test-slave-xb9n krb5kdc[18991](info): commencing operation
krb5kdc: starting...
W20260619 16:57:49.805174 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:57:49 dist-test-slave-xb9n krb5kdc[18991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:57:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 16:57:49.915544 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46159
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:49.992318 19004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:49.992501 19004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:49.992522 19004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:49.993647 19004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:49.993687 19004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:49.993700 19004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:49.993713 19004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:49.993762 19004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:49.995141 19004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46159
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:49.995359 19004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:49.995584 19004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:50.000257 19009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:50.000257 19012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:50.000283 19010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:50.000422 19004 server_base.cc:1034] running on GCE node
I20260619 16:57:50.000698 19004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:50.000965 19004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:50.002149 19004 hybrid_clock.cc:648] HybridClock initialized: now 1781888270002137 us; error 37 us; skew 500 ppm
Jun 19 16:57:49 dist-test-slave-xb9n krb5kdc[18991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888269, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:50.004622 19004 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 16:57:50.005110 19004 webserver.cc:460] Webserver started at http://127.0.0.1:35895/ using document root <none> and password file <none>
I20260619 16:57:50.005342 19004 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:50.005455 19004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:50.005609 19004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:50.006712 19004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "7e3df1968f784c369e7d6a61cd323412"
format_stamp: "Formatted at 2026-06-19 16:57:50 on dist-test-slave-xb9n"
server_key: "e66112d41eed0f64eebf2acc8ccc6a59"
server_key_iv: "c9c85eb0822d8cc27c1f9cd452630e87"
server_key_version: "encryptionkey@0"
I20260619 16:57:50.007090 19004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "7e3df1968f784c369e7d6a61cd323412"
format_stamp: "Formatted at 2026-06-19 16:57:50 on dist-test-slave-xb9n"
server_key: "e66112d41eed0f64eebf2acc8ccc6a59"
server_key_iv: "c9c85eb0822d8cc27c1f9cd452630e87"
server_key_version: "encryptionkey@0"
I20260619 16:57:50.008431 19004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:57:50.009238 19019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:50.009436 19004 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:50.009536 19004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "7e3df1968f784c369e7d6a61cd323412"
format_stamp: "Formatted at 2026-06-19 16:57:50 on dist-test-slave-xb9n"
server_key: "e66112d41eed0f64eebf2acc8ccc6a59"
server_key_iv: "c9c85eb0822d8cc27c1f9cd452630e87"
server_key_version: "encryptionkey@0"
I20260619 16:57:50.009613 19004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:50.019615 19004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:50.021571 19004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:50.021719 19004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:50.025471 19004 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46159
I20260619 16:57:50.025509 19071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46159 every 8 connection(s)
I20260619 16:57:50.025854 19004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:50.026684 19072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:50.028944 19072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412: Bootstrap starting.
I20260619 16:57:50.029390 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 19004
I20260619 16:57:50.029508 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:50.029704 19072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:50.029747 31314 external_mini_cluster.cc:1428] Setting key cc4b38fe34c7254ec49500e6a6e64073
I20260619 16:57:50.030014 19072 log.cc:824] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:50.030767 19072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412: No bootstrap required, opened a new log
I20260619 16:57:50.033082 19072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e3df1968f784c369e7d6a61cd323412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46159 } }
I20260619 16:57:50.033246 19072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:50.033275 19072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e3df1968f784c369e7d6a61cd323412, State: Initialized, Role: FOLLOWER
I20260619 16:57:50.033460 19072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e3df1968f784c369e7d6a61cd323412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46159 } }
I20260619 16:57:50.033557 19072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:50.033591 19072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:50.033638 19072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:50.034173 19072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e3df1968f784c369e7d6a61cd323412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46159 } }
I20260619 16:57:50.034255 19072 leader_election.cc:302] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e3df1968f784c369e7d6a61cd323412; no voters:
I20260619 16:57:50.034438 19072 leader_election.cc:288] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:50.034492 19077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:50.034642 19077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [term 1 LEADER]: Becoming Leader. State: Replica: 7e3df1968f784c369e7d6a61cd323412, State: Running, Role: LEADER
I20260619 16:57:50.034675 19072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:50.034731 19077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e3df1968f784c369e7d6a61cd323412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46159 } }
I20260619 16:57:50.035125 19078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e3df1968f784c369e7d6a61cd323412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e3df1968f784c369e7d6a61cd323412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46159 } } }
I20260619 16:57:50.035218 19078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:50.035440 19079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e3df1968f784c369e7d6a61cd323412. Latest consensus state: current_term: 1 leader_uuid: "7e3df1968f784c369e7d6a61cd323412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e3df1968f784c369e7d6a61cd323412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46159 } } }
I20260619 16:57:50.035558 19079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:50.035732 19085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:50.036240 19085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 16:57:50 dist-test-slave-xb9n krb5kdc[18991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 16:57:50.037882 19085 catalog_manager.cc:1347] Generated new cluster ID: 936e3d3510a2472a9a4448de1337cebd
I20260619 16:57:50.037932 19085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:50.042017 19076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:50.030276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55021 (local address 127.0.0.1:46159)
0619 16:57:50.030441 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:50.030450 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:50.030482 (+ 32us) server_negotiation.cc:407] Connection header received
0619 16:57:50.031066 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:50.031075 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:50.031136 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:50.031286 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:50.031956 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:50.033445 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:50.034316 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:50.034955 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:50.035421 (+ 466us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:50.038407 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:50.038411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:50.038411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:50.038421 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:50.040322 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:50.040785 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:50.040788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:50.040788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:50.040890 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:50.041132 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:50.041132 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:50.041133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:50.041426 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:50.041575 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:50.041729 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 16:57:50.041831 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260619 16:57:50.042984 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:57:50.051766 19076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:50.044461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39103 (local address 127.0.0.1:46159)
0619 16:57:50.044498 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:50.044500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:50.044509 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:57:50.044535 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:50.044536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:50.044544 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:50.044613 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:50.045176 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:50.045933 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:50.046894 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:50.047234 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:50.047310 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:50.048702 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:50.048703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:50.048703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:50.048710 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:50.050425 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:50.050829 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:50.050830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:50.050831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:50.050891 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:50.051105 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:50.051106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:50.051106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:50.051246 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:50.051352 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:50.051582 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 16:57:50.051663 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:57:50.053217 19085 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:50.053756 19085 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:50.063678 19085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e3df1968f784c369e7d6a61cd323412: Generated new TSK 0
I20260619 16:57:50.063892 19085 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:57:50.070616 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 19004
W20260619 16:57:50.074764 18989 connection.cc:535] client connection to 127.0.0.1:46159 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46159): Connection reset by peer (error 104)
2026-06-19T16:57:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:57:50 dist-test-slave-xb9n krb5kdc[19103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:57:50 dist-test-slave-xb9n krb5kdc[19103](info): set up 2 sockets
Jun 19 16:57:50 dist-test-slave-xb9n krb5kdc[19103](info): commencing operation
krb5kdc: starting...
W20260619 16:57:51.101194 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:57:51 dist-test-slave-xb9n krb5kdc[19103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:57:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 16:57:51.209735 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38541
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:51.287513 19116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:51.287703 19116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:51.287725 19116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:51.288785 19116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:51.288821 19116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:51.288836 19116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:51.288848 19116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:51.288861 19116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:51.290203 19116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38541
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:51.290364 19116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:51.290517 19116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:51.295050 19121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:51.295051 19124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:51.295053 19122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:51.295135 19116 server_base.cc:1034] running on GCE node
I20260619 16:57:51.295521 19116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:51.295804 19116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:51.296953 19116 hybrid_clock.cc:648] HybridClock initialized: now 1781888271296943 us; error 27 us; skew 500 ppm
Jun 19 16:57:51 dist-test-slave-xb9n krb5kdc[19103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888271, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:51.299216 19116 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 16:57:51.299638 19116 webserver.cc:460] Webserver started at http://127.0.0.1:41241/ using document root <none> and password file <none>
I20260619 16:57:51.299849 19116 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:51.299908 19116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:51.300041 19116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:51.301011 19116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "f90a67f2f4d74dddbde4fba4baca5851"
format_stamp: "Formatted at 2026-06-19 16:57:51 on dist-test-slave-xb9n"
server_key: "a24cf054c07b5340200e24a32c2a1616"
server_key_iv: "569b64b2b4a6eeceecd67d86640ee855"
server_key_version: "encryptionkey@0"
I20260619 16:57:51.301550 19116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "f90a67f2f4d74dddbde4fba4baca5851"
format_stamp: "Formatted at 2026-06-19 16:57:51 on dist-test-slave-xb9n"
server_key: "a24cf054c07b5340200e24a32c2a1616"
server_key_iv: "569b64b2b4a6eeceecd67d86640ee855"
server_key_version: "encryptionkey@0"
I20260619 16:57:51.302831 19116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:57:51.303580 19131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:51.303743 19116 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:57:51.303833 19116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "f90a67f2f4d74dddbde4fba4baca5851"
format_stamp: "Formatted at 2026-06-19 16:57:51 on dist-test-slave-xb9n"
server_key: "a24cf054c07b5340200e24a32c2a1616"
server_key_iv: "569b64b2b4a6eeceecd67d86640ee855"
server_key_version: "encryptionkey@0"
I20260619 16:57:51.303910 19116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:51.321188 19116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:51.322803 19116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:51.322948 19116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:51.326773 19116 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38541
I20260619 16:57:51.326819 19183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38541 every 8 connection(s)
I20260619 16:57:51.327154 19116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:51.327898 19184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:51.330037 19184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851: Bootstrap starting.
I20260619 16:57:51.330534 19184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:51.330773 19184 log.cc:824] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:51.331368 19184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851: No bootstrap required, opened a new log
I20260619 16:57:51.332763 19184 raft_consensus.cc:348] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90a67f2f4d74dddbde4fba4baca5851" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38541 } }
I20260619 16:57:51.332885 19184 raft_consensus.cc:374] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:51.332906 19184 raft_consensus.cc:729] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f90a67f2f4d74dddbde4fba4baca5851, State: Initialized, Role: FOLLOWER
I20260619 16:57:51.332996 19184 consensus_queue.cc:260] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90a67f2f4d74dddbde4fba4baca5851" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38541 } }
I20260619 16:57:51.333042 19184 raft_consensus.cc:388] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:51.333061 19184 raft_consensus.cc:482] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:51.333086 19184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:51.333652 19184 raft_consensus.cc:504] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90a67f2f4d74dddbde4fba4baca5851" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38541 } }
I20260619 16:57:51.333729 19184 leader_election.cc:302] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f90a67f2f4d74dddbde4fba4baca5851; no voters:
I20260619 16:57:51.333786 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 19116
I20260619 16:57:51.333900 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:51.333918 19184 leader_election.cc:288] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:51.333949 19187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:51.334076 19187 raft_consensus.cc:686] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [term 1 LEADER]: Becoming Leader. State: Replica: f90a67f2f4d74dddbde4fba4baca5851, State: Running, Role: LEADER
I20260619 16:57:51.334141 31314 external_mini_cluster.cc:1428] Setting key 8866da7eea51796a0a240e8906003c3c
I20260619 16:57:51.334162 19184 sys_catalog.cc:564] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:51.334152 19187 consensus_queue.cc:237] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90a67f2f4d74dddbde4fba4baca5851" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38541 } }
I20260619 16:57:51.334522 19187 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f90a67f2f4d74dddbde4fba4baca5851. Latest consensus state: current_term: 1 leader_uuid: "f90a67f2f4d74dddbde4fba4baca5851" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90a67f2f4d74dddbde4fba4baca5851" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38541 } } }
I20260619 16:57:51.334534 19189 sys_catalog.cc:455] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f90a67f2f4d74dddbde4fba4baca5851" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f90a67f2f4d74dddbde4fba4baca5851" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38541 } } }
I20260619 16:57:51.334610 19187 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:51.334645 19189 sys_catalog.cc:458] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:51.335021 19192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:51.335522 19192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:51.337863 19192 catalog_manager.cc:1347] Generated new cluster ID: f95757a594b64b47b7447b8842b5299e
I20260619 16:57:51.337945 19192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:51.340292 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:51.334962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55561 (local address 127.0.0.1:38541)
0619 16:57:51.335401 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:51.335410 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:51.335439 (+ 29us) server_negotiation.cc:407] Connection header received
0619 16:57:51.335550 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:51.335557 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:51.335629 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:51.335816 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:51.336093 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:51.337122 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:51.338235 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:51.338889 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:51.339001 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:51.339645 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:51.339648 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:51.339648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:51.339849 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:51.339887 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:51.339901 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:51.339986 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:51.340062 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 16:57:51.340085 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":78,"threads_started":1}
I20260619 16:57:51.341176 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:57:51.346032 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:51.342537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48465 (local address 127.0.0.1:38541)
0619 16:57:51.342572 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:51.342573 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:51.342836 (+ 263us) server_negotiation.cc:407] Connection header received
0619 16:57:51.342885 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:51.342886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:51.342897 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:51.342978 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:51.343097 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:51.343832 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:51.344773 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:51.345088 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:51.345162 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:51.345803 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:51.345804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:51.345804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:51.345808 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:51.345821 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:51.345826 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:51.345888 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:51.345952 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 16:57:51.345964 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 16:57:51.361865 19192 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:51.362326 19192 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:51.371203 19192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f90a67f2f4d74dddbde4fba4baca5851: Generated new TSK 0
I20260619 16:57:51.371356 19192 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:57:51.380506 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 19116
W20260619 16:57:51.384102 19101 connection.cc:535] client connection to 127.0.0.1:38541 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38541): Connection reset by peer (error 104)
2026-06-19T16:57:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:57:51 dist-test-slave-xb9n krb5kdc[19215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:57:51 dist-test-slave-xb9n krb5kdc[19215](info): set up 2 sockets
Jun 19 16:57:51 dist-test-slave-xb9n krb5kdc[19215](info): commencing operation
krb5kdc: starting...
W20260619 16:57:52.409327 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:57:52 dist-test-slave-xb9n krb5kdc[19215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:57:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 16:57:52.520334 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35661
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:52.599421 19228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:52.599606 19228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:52.599628 19228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:52.600736 19228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:52.600778 19228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:52.600792 19228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:52.600805 19228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:52.600817 19228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:52.602299 19228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35661
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:52.602530 19228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:52.602754 19228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:52.607439 19236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:52.607426 19233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:52.607426 19234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:52.607806 19228 server_base.cc:1034] running on GCE node
I20260619 16:57:52.608040 19228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:52.608326 19228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:52.609457 19228 hybrid_clock.cc:648] HybridClock initialized: now 1781888272609438 us; error 44 us; skew 500 ppm
Jun 19 16:57:52 dist-test-slave-xb9n krb5kdc[19215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888272, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:52.611883 19228 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 16:57:52.612251 19228 webserver.cc:460] Webserver started at http://127.0.0.1:40605/ using document root <none> and password file <none>
I20260619 16:57:52.612471 19228 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:52.612515 19228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:52.612673 19228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:52.613772 19228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "7279b2af84e544eda7dcabaa28e63e3d"
format_stamp: "Formatted at 2026-06-19 16:57:52 on dist-test-slave-xb9n"
server_key: "e1b64bc887f442b49162033396cad434"
server_key_iv: "5c76eec3903dc62a63eb7731357522f9"
server_key_version: "encryptionkey@0"
I20260619 16:57:52.614142 19228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "7279b2af84e544eda7dcabaa28e63e3d"
format_stamp: "Formatted at 2026-06-19 16:57:52 on dist-test-slave-xb9n"
server_key: "e1b64bc887f442b49162033396cad434"
server_key_iv: "5c76eec3903dc62a63eb7731357522f9"
server_key_version: "encryptionkey@0"
I20260619 16:57:52.615453 19228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:57:52.616221 19243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:52.616413 19228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:52.616511 19228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "7279b2af84e544eda7dcabaa28e63e3d"
format_stamp: "Formatted at 2026-06-19 16:57:52 on dist-test-slave-xb9n"
server_key: "e1b64bc887f442b49162033396cad434"
server_key_iv: "5c76eec3903dc62a63eb7731357522f9"
server_key_version: "encryptionkey@0"
I20260619 16:57:52.616600 19228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:52.619736 19228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:52.621790 19228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:52.621948 19228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:52.625896 19228 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35661
I20260619 16:57:52.625926 19295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35661 every 8 connection(s)
I20260619 16:57:52.626232 19228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:52.627188 19296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:52.629238 19296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d: Bootstrap starting.
I20260619 16:57:52.629806 19296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:52.630026 19296 log.cc:824] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:52.630626 19296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d: No bootstrap required, opened a new log
I20260619 16:57:52.632083 19296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7279b2af84e544eda7dcabaa28e63e3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35661 } }
I20260619 16:57:52.632215 19296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:52.632237 19296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7279b2af84e544eda7dcabaa28e63e3d, State: Initialized, Role: FOLLOWER
I20260619 16:57:52.632329 19296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7279b2af84e544eda7dcabaa28e63e3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35661 } }
I20260619 16:57:52.632376 19296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:52.632395 19296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:52.632419 19296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:52.632863 19296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7279b2af84e544eda7dcabaa28e63e3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35661 } }
I20260619 16:57:52.632944 19296 leader_election.cc:302] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7279b2af84e544eda7dcabaa28e63e3d; no voters:
I20260619 16:57:52.633083 19296 leader_election.cc:288] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:52.633116 19299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:52.633234 19299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [term 1 LEADER]: Becoming Leader. State: Replica: 7279b2af84e544eda7dcabaa28e63e3d, State: Running, Role: LEADER
I20260619 16:57:52.633340 19296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:52.633399 19299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7279b2af84e544eda7dcabaa28e63e3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35661 } }
I20260619 16:57:52.633812 19301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7279b2af84e544eda7dcabaa28e63e3d. Latest consensus state: current_term: 1 leader_uuid: "7279b2af84e544eda7dcabaa28e63e3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7279b2af84e544eda7dcabaa28e63e3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35661 } } }
I20260619 16:57:52.633899 19300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7279b2af84e544eda7dcabaa28e63e3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7279b2af84e544eda7dcabaa28e63e3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35661 } } }
I20260619 16:57:52.634021 19300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:52.633967 19301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:52.634416 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 19228
I20260619 16:57:52.634433 19308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:52.634546 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:52.634792 31314 external_mini_cluster.cc:1428] Setting key cb9c61e2adde689ebb482919bce0fe1e
I20260619 16:57:52.634955 19308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:52.636915 19308 catalog_manager.cc:1347] Generated new cluster ID: 6cff16f31df14b0aa2175f1709fc7311
I20260619 16:57:52.636968 19308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:52.637482 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:52.635217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57977 (local address 127.0.0.1:35661)
0619 16:57:52.635408 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:52.635417 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:52.635981 (+ 564us) server_negotiation.cc:407] Connection header received
0619 16:57:52.636158 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:52.636165 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:52.636197 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:52.636327 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:52.637072 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:52.637076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:52.637076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:52.637080 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:52.637119 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:52.637133 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:52.637209 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:52.637281 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 16:57:52.637314 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
I20260619 16:57:52.638360 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:57:52.645085 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:52.643473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38165 (local address 127.0.0.1:35661)
0619 16:57:52.644145 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:52.644147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:52.644155 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:57:52.644173 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:52.644174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:52.644178 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:52.644227 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:52.644836 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:52.644837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:52.644837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:52.644841 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:52.644854 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:52.644859 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:52.644888 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:52.645011 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 16:57:52.645024 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652}
I20260619 16:57:52.651476 19308 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:52.651616 19308 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:52.655175 19308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7279b2af84e544eda7dcabaa28e63e3d: Generated new TSK 0
I20260619 16:57:52.655313 19308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:57:52.657037 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 19228
2026-06-19T16:57:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1275 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:57:52 dist-test-slave-xb9n krb5kdc[19327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:57:52 dist-test-slave-xb9n krb5kdc[19327](info): set up 2 sockets
Jun 19 16:57:52 dist-test-slave-xb9n krb5kdc[19327](info): commencing operation
krb5kdc: starting...
W20260619 16:57:53.697257 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:57:53 dist-test-slave-xb9n krb5kdc[19327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:57:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 16:57:53.806596 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43297
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:53.883620 19340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:53.883849 19340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:53.883886 19340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:53.885001 19340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:53.885072 19340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:53.885097 19340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:53.885116 19340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:53.885147 19340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:53.886564 19340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:53.886827 19340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:53.887038 19340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:53.891855 19346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:53.891840 19348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:53.891866 19340 server_base.cc:1034] running on GCE node
W20260619 16:57:53.891841 19345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:53.892266 19340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:53.892483 19340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:53.893616 19340 hybrid_clock.cc:648] HybridClock initialized: now 1781888273893601 us; error 46 us; skew 500 ppm
Jun 19 16:57:53 dist-test-slave-xb9n krb5kdc[19327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888273, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:53.895920 19340 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 16:57:53.896324 19340 webserver.cc:460] Webserver started at http://127.0.0.1:41223/ using document root <none> and password file <none>
I20260619 16:57:53.896531 19340 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:53.896574 19340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:53.896724 19340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:53.897784 19340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "2b406f824faf4cd7b87fe89ae0932ee5"
format_stamp: "Formatted at 2026-06-19 16:57:53 on dist-test-slave-xb9n"
server_key: "d62d9a15bf2c287a7769dc5b273e9933"
server_key_iv: "6e320c5518f38423c00b37d1ee514280"
server_key_version: "encryptionkey@0"
I20260619 16:57:53.898129 19340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "2b406f824faf4cd7b87fe89ae0932ee5"
format_stamp: "Formatted at 2026-06-19 16:57:53 on dist-test-slave-xb9n"
server_key: "d62d9a15bf2c287a7769dc5b273e9933"
server_key_iv: "6e320c5518f38423c00b37d1ee514280"
server_key_version: "encryptionkey@0"
I20260619 16:57:53.899411 19340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:57:53.900388 19355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:53.900614 19340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:57:53.900728 19340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "2b406f824faf4cd7b87fe89ae0932ee5"
format_stamp: "Formatted at 2026-06-19 16:57:53 on dist-test-slave-xb9n"
server_key: "d62d9a15bf2c287a7769dc5b273e9933"
server_key_iv: "6e320c5518f38423c00b37d1ee514280"
server_key_version: "encryptionkey@0"
I20260619 16:57:53.900799 19340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:53.923247 19340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:53.925073 19340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:53.925246 19340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:53.928817 19340 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43297
I20260619 16:57:53.928871 19407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43297 every 8 connection(s)
I20260619 16:57:53.929220 19340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:53.930080 19408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:53.930492 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 19340
I20260619 16:57:53.930610 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:53.930835 31314 external_mini_cluster.cc:1428] Setting key fc07b03f950602505d43f6710d14b319
I20260619 16:57:53.932696 19408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5: Bootstrap starting.
I20260619 16:57:53.933264 19408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:53.933593 19408 log.cc:824] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:53.934370 19408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5: No bootstrap required, opened a new log
I20260619 16:57:53.936789 19408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b406f824faf4cd7b87fe89ae0932ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43297 } }
I20260619 16:57:53.936965 19408 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:53.937011 19408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b406f824faf4cd7b87fe89ae0932ee5, State: Initialized, Role: FOLLOWER
I20260619 16:57:53.937150 19408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b406f824faf4cd7b87fe89ae0932ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43297 } }
I20260619 16:57:53.937237 19408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:53.937259 19408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:53.937291 19408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 16:57:53 dist-test-slave-xb9n krb5kdc[19327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 16:57:53.937875 19408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b406f824faf4cd7b87fe89ae0932ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43297 } }
I20260619 16:57:53.937991 19408 leader_election.cc:302] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b406f824faf4cd7b87fe89ae0932ee5; no voters:
I20260619 16:57:53.938159 19408 leader_election.cc:288] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:53.938321 19413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:53.938402 19408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:53.938612 19413 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [term 1 LEADER]: Becoming Leader. State: Replica: 2b406f824faf4cd7b87fe89ae0932ee5, State: Running, Role: LEADER
I20260619 16:57:53.938757 19413 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b406f824faf4cd7b87fe89ae0932ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43297 } }
I20260619 16:57:53.939109 19415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b406f824faf4cd7b87fe89ae0932ee5. Latest consensus state: current_term: 1 leader_uuid: "2b406f824faf4cd7b87fe89ae0932ee5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b406f824faf4cd7b87fe89ae0932ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43297 } } }
I20260619 16:57:53.939188 19415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:53.939419 19422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:53.939417 19414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b406f824faf4cd7b87fe89ae0932ee5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b406f824faf4cd7b87fe89ae0932ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43297 } } }
I20260619 16:57:53.940124 19414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:53.940907 19422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:53.941996 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:53.931343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42407 (local address 127.0.0.1:43297)
0619 16:57:53.931517 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:53.931532 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:53.931555 (+ 23us) server_negotiation.cc:407] Connection header received
0619 16:57:53.931973 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:53.931979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:53.932022 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:53.932173 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:53.932812 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:53.933811 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:53.934925 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:53.935322 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:53.935398 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:53.938312 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:53.938317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:53.938317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:53.938330 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:53.940479 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:53.940812 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:53.940815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:53.940816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:53.940905 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:53.941117 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:53.941118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:53.941118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:53.941342 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:53.941530 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:53.941622 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 16:57:53.941763 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260619 16:57:53.942406 19422 catalog_manager.cc:1347] Generated new cluster ID: c2132c426d0d4ebaafc9e9f0e501164c
I20260619 16:57:53.942454 19422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:53.942873 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:57:53.948279 19422 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:53.948726 19422 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:53.951790 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:53.944462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41687 (local address 127.0.0.1:43297)
0619 16:57:53.944493 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:53.944494 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:53.944653 (+ 159us) server_negotiation.cc:407] Connection header received
0619 16:57:53.944713 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:53.944714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:53.944722 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:53.944772 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:53.945396 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:53.946129 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:53.947149 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:53.947449 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:53.947516 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:53.948879 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:53.948880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:53.948880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:53.948886 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:53.950423 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:53.950795 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:53.950796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:53.950796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:53.950846 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:53.951026 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:53.951026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:53.951027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:53.951145 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:53.951227 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:53.951629 (+ 402us) server_negotiation.cc:299] Negotiation successful
0619 16:57:53.951710 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 16:57:53.956480 19422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b406f824faf4cd7b87fe89ae0932ee5: Generated new TSK 0
I20260619 16:57:53.956640 19422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:57:53.960588 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 19340
2026-06-19T16:57:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:57:54 dist-test-slave-xb9n krb5kdc[19439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:57:54 dist-test-slave-xb9n krb5kdc[19439](info): set up 2 sockets
Jun 19 16:57:54 dist-test-slave-xb9n krb5kdc[19439](info): commencing operation
krb5kdc: starting...
W20260619 16:57:54.989416 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:57:55 dist-test-slave-xb9n krb5kdc[19439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:57:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 16:57:55.099689 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:55.178263 19452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:55.178472 19452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:55.178505 19452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:55.179603 19452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:55.179662 19452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:55.179687 19452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:55.179708 19452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:55.179739 19452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:55.181167 19452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:55.181437 19452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:55.181650 19452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:55.186091 19457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:55.186115 19460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:55.186192 19458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:55.186224 19452 server_base.cc:1034] running on GCE node
I20260619 16:57:55.186628 19452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:55.186908 19452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:55.188072 19452 hybrid_clock.cc:648] HybridClock initialized: now 1781888275188056 us; error 28 us; skew 500 ppm
Jun 19 16:57:55 dist-test-slave-xb9n krb5kdc[19439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888275, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:55.190464 19452 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 16:57:55.190886 19452 webserver.cc:460] Webserver started at http://127.0.0.1:39357/ using document root <none> and password file <none>
I20260619 16:57:55.191087 19452 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:55.191149 19452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:55.191298 19452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:55.192235 19452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "7883570005314657a82d759524c8fd52"
format_stamp: "Formatted at 2026-06-19 16:57:55 on dist-test-slave-xb9n"
server_key: "ad9ace9956c3bfa8a55173a6023b761c"
server_key_iv: "7f7b035d199c57e2e72d03d376090f97"
server_key_version: "encryptionkey@0"
I20260619 16:57:55.192560 19452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "7883570005314657a82d759524c8fd52"
format_stamp: "Formatted at 2026-06-19 16:57:55 on dist-test-slave-xb9n"
server_key: "ad9ace9956c3bfa8a55173a6023b761c"
server_key_iv: "7f7b035d199c57e2e72d03d376090f97"
server_key_version: "encryptionkey@0"
I20260619 16:57:55.193856 19452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:57:55.194511 19467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:55.194677 19452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:55.194777 19452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "7883570005314657a82d759524c8fd52"
format_stamp: "Formatted at 2026-06-19 16:57:55 on dist-test-slave-xb9n"
server_key: "ad9ace9956c3bfa8a55173a6023b761c"
server_key_iv: "7f7b035d199c57e2e72d03d376090f97"
server_key_version: "encryptionkey@0"
I20260619 16:57:55.194849 19452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:55.216225 19452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:55.217939 19452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:55.218092 19452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:55.221769 19452 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43715
I20260619 16:57:55.221797 19519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43715 every 8 connection(s)
I20260619 16:57:55.222111 19452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:55.222831 19520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:55.223742 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 19452
I20260619 16:57:55.223856 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:55.224038 31314 external_mini_cluster.cc:1428] Setting key 87b0e4b37ce995828f7b598c28115c36
I20260619 16:57:55.225512 19520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52: Bootstrap starting.
I20260619 16:57:55.226078 19520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:55.226361 19520 log.cc:824] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:55.226997 19520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52: No bootstrap required, opened a new log
I20260619 16:57:55.228806 19520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7883570005314657a82d759524c8fd52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43715 } }
I20260619 16:57:55.228953 19520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:55.229001 19520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7883570005314657a82d759524c8fd52, State: Initialized, Role: FOLLOWER
I20260619 16:57:55.229182 19520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7883570005314657a82d759524c8fd52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43715 } }
I20260619 16:57:55.229314 19520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:55.229354 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:55.224533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53433 (local address 127.0.0.1:43715)
0619 16:57:55.224706 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:55.224720 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:55.224742 (+ 22us) server_negotiation.cc:407] Connection header received
0619 16:57:55.225258 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:55.225263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:55.225305 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:55.225428 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:55.225664 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:55.226620 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:55.227444 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:55.227847 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:55.227957 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:55.228654 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:55.228660 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:55.228660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:55.228809 (+ 149us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:55.228835 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:55.228845 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:55.228920 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:55.229031 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 16:57:55.229098 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260619 16:57:55.229382 19520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:55.229460 19520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:55.230077 19520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7883570005314657a82d759524c8fd52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43715 } }
I20260619 16:57:55.230199 19520 leader_election.cc:302] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7883570005314657a82d759524c8fd52; no voters:
I20260619 16:57:55.230237 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:57:55.230348 19520 leader_election.cc:288] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:55.230377 19525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:55.230480 19525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [term 1 LEADER]: Becoming Leader. State: Replica: 7883570005314657a82d759524c8fd52, State: Running, Role: LEADER
I20260619 16:57:55.230583 19520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:55.230600 19525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7883570005314657a82d759524c8fd52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43715 } }
I20260619 16:57:55.230927 19525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7883570005314657a82d759524c8fd52. Latest consensus state: current_term: 1 leader_uuid: "7883570005314657a82d759524c8fd52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7883570005314657a82d759524c8fd52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43715 } } }
I20260619 16:57:55.230939 19526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7883570005314657a82d759524c8fd52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7883570005314657a82d759524c8fd52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43715 } } }
I20260619 16:57:55.231177 19526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:55.231093 19525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52 [sys.catalog]: This master's current role is: LEADER
W20260619 16:57:55.232324 19544 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:57:55.232375 19544 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:57:55.232427 19537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:55.232614 19537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:55.234009 19537 catalog_manager.cc:1347] Generated new cluster ID: c3177551c2ef45cb87cfb0db88b6e756
I20260619 16:57:55.234054 19537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:55.236709 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:55.232291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47775 (local address 127.0.0.1:43715)
0619 16:57:55.232326 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:55.232328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:55.233506 (+ 1178us) server_negotiation.cc:407] Connection header received
0619 16:57:55.233610 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:55.233611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:55.233621 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:55.233682 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:55.233824 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:55.234511 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:55.235513 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:55.235792 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:55.235845 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:55.236493 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:55.236494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:55.236494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:55.236498 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:55.236509 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:55.236513 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:55.236560 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:55.236624 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 16:57:55.236643 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:57:55.254021 19537 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:55.254522 19537 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:55.257989 19537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7883570005314657a82d759524c8fd52: Generated new TSK 0
I20260619 16:57:55.258180 19537 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:57:55.263386 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 19452
2026-06-19T16:57:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1301 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:57:55 dist-test-slave-xb9n krb5kdc[19551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:57:55 dist-test-slave-xb9n krb5kdc[19551](info): set up 2 sockets
Jun 19 16:57:55 dist-test-slave-xb9n krb5kdc[19551](info): commencing operation
krb5kdc: starting...
W20260619 16:57:56.285307 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:57:56 dist-test-slave-xb9n krb5kdc[19551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:57:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 16:57:56.394387 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43989
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:56.472322 19564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:56.472505 19564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:56.472527 19564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:56.473587 19564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:56.473621 19564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:56.473641 19564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:56.473654 19564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:56.473666 19564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:56.475018 19564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:56.475193 19564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:56.475347 19564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:56.480070 19572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:56.480118 19569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:56.480080 19570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:56.480268 19564 server_base.cc:1034] running on GCE node
I20260619 16:57:56.480589 19564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:56.480841 19564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:56.481977 19564 hybrid_clock.cc:648] HybridClock initialized: now 1781888276481967 us; error 28 us; skew 500 ppm
Jun 19 16:57:56 dist-test-slave-xb9n krb5kdc[19551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888276, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:56.484268 19564 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 16:57:56.484716 19564 webserver.cc:460] Webserver started at http://127.0.0.1:33201/ using document root <none> and password file <none>
I20260619 16:57:56.484941 19564 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:56.485004 19564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:56.485143 19564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:56.486176 19564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "dfc3c8c969a34e46b4e15b24243261b9"
format_stamp: "Formatted at 2026-06-19 16:57:56 on dist-test-slave-xb9n"
server_key: "e162e45cda0426db8afeffa017f7bbac"
server_key_iv: "93a5478f7f0173bb177b28f5ecf078a8"
server_key_version: "encryptionkey@0"
I20260619 16:57:56.486518 19564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "dfc3c8c969a34e46b4e15b24243261b9"
format_stamp: "Formatted at 2026-06-19 16:57:56 on dist-test-slave-xb9n"
server_key: "e162e45cda0426db8afeffa017f7bbac"
server_key_iv: "93a5478f7f0173bb177b28f5ecf078a8"
server_key_version: "encryptionkey@0"
I20260619 16:57:56.487769 19564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:57:56.488464 19579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:56.488631 19564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:57:56.488727 19564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "dfc3c8c969a34e46b4e15b24243261b9"
format_stamp: "Formatted at 2026-06-19 16:57:56 on dist-test-slave-xb9n"
server_key: "e162e45cda0426db8afeffa017f7bbac"
server_key_iv: "93a5478f7f0173bb177b28f5ecf078a8"
server_key_version: "encryptionkey@0"
I20260619 16:57:56.488806 19564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:56.492027 19564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:56.493861 19564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:56.494004 19564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:56.497812 19564 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43989
I20260619 16:57:56.497839 19631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43989 every 8 connection(s)
I20260619 16:57:56.498124 19564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:56.499006 19632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:56.501026 19632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9: Bootstrap starting.
I20260619 16:57:56.501685 19632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:56.501968 19632 log.cc:824] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:56.502619 19632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9: No bootstrap required, opened a new log
I20260619 16:57:56.504660 19632 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc3c8c969a34e46b4e15b24243261b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43989 } }
I20260619 16:57:56.504781 19632 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:56.504802 19632 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc3c8c969a34e46b4e15b24243261b9, State: Initialized, Role: FOLLOWER
I20260619 16:57:56.504954 19632 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc3c8c969a34e46b4e15b24243261b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43989 } }
I20260619 16:57:56.505009 19632 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:56.505079 19632 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:56.505128 19632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:56.505725 19632 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc3c8c969a34e46b4e15b24243261b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43989 } }
I20260619 16:57:56.505831 19632 leader_election.cc:302] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfc3c8c969a34e46b4e15b24243261b9; no voters:
I20260619 16:57:56.505980 19632 leader_election.cc:288] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:56.506026 19635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:56.506210 19635 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [term 1 LEADER]: Becoming Leader. State: Replica: dfc3c8c969a34e46b4e15b24243261b9, State: Running, Role: LEADER
I20260619 16:57:56.506222 19632 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:56.506312 19635 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc3c8c969a34e46b4e15b24243261b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43989 } }
I20260619 16:57:56.506667 19637 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfc3c8c969a34e46b4e15b24243261b9. Latest consensus state: current_term: 1 leader_uuid: "dfc3c8c969a34e46b4e15b24243261b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc3c8c969a34e46b4e15b24243261b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43989 } } }
I20260619 16:57:56.506666 19636 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfc3c8c969a34e46b4e15b24243261b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc3c8c969a34e46b4e15b24243261b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43989 } } }
I20260619 16:57:56.506794 19637 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:56.506858 19636 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:56.507223 19639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:56.507712 19639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:56.508248 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 19564
I20260619 16:57:56.508373 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:56.508561 31314 external_mini_cluster.cc:1428] Setting key cb48ce76f02e0cf1a0d4d58a3ddd9186
I20260619 16:57:56.510362 19639 catalog_manager.cc:1347] Generated new cluster ID: 7860cea6e64b42d7a0625d52b3080d5d
I20260619 16:57:56.510411 19639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:56.514001 19654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:56.508998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42335 (local address 127.0.0.1:43989)
0619 16:57:56.509481 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:56.509488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:56.509512 (+ 24us) server_negotiation.cc:407] Connection header received
0619 16:57:56.509622 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:56.509627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:56.509649 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:56.509756 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:56.513590 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:56.513593 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:56.513593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:56.513597 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:56.513661 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:56.513672 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:56.513741 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:56.513814 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 16:57:56.513853 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":84,"threads_started":1}
I20260619 16:57:56.514855 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:57:56.522068 19639 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:56.522233 19639 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:56.529768 19654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:56.517612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55613 (local address 127.0.0.1:43989)
0619 16:57:56.517652 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:56.517654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:56.521557 (+ 3903us) server_negotiation.cc:407] Connection header received
0619 16:57:56.521605 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:56.521606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:56.521611 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:56.521677 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:56.529552 (+ 7875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:56.529553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:56.529554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:56.529558 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:56.529574 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:56.529578 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:56.529643 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:56.529693 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 16:57:56.529706 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:57:56.531273 19639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfc3c8c969a34e46b4e15b24243261b9: Generated new TSK 0
I20260619 16:57:56.531425 19639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:57:56.537218 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 19564
2026-06-19T16:57:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1273 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:57:56 dist-test-slave-xb9n krb5kdc[19663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:57:56 dist-test-slave-xb9n krb5kdc[19663](info): set up 2 sockets
Jun 19 16:57:56 dist-test-slave-xb9n krb5kdc[19663](info): commencing operation
krb5kdc: starting...
W20260619 16:57:57.573302 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:57:57 dist-test-slave-xb9n krb5kdc[19663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:57:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 16:57:57.683449 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40827
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:57.761184 19676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:57.761410 19676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:57.761456 19676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:57.762571 19676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:57.762636 19676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:57.762669 19676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:57.762701 19676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:57.762723 19676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:57.764086 19676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:57.764334 19676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:57.764539 19676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:57.769331 19682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:57.769340 19684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:57.769340 19681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:57.769529 19676 server_base.cc:1034] running on GCE node
I20260619 16:57:57.769802 19676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:57.770088 19676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:57.771262 19676 hybrid_clock.cc:648] HybridClock initialized: now 1781888277771245 us; error 37 us; skew 500 ppm
Jun 19 16:57:57 dist-test-slave-xb9n krb5kdc[19663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888277, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:57.777275 19676 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 16:57:57.777747 19676 webserver.cc:460] Webserver started at http://127.0.0.1:39663/ using document root <none> and password file <none>
I20260619 16:57:57.777956 19676 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:57.778028 19676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:57.778175 19676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:57.779189 19676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "bb732b4ca4424c55933e398ea54974de"
format_stamp: "Formatted at 2026-06-19 16:57:57 on dist-test-slave-xb9n"
server_key: "a411808715154b97b41caa0537cb59e7"
server_key_iv: "cc1da954c0d37ab07124b9e68f5ae2cb"
server_key_version: "encryptionkey@0"
I20260619 16:57:57.779537 19676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "bb732b4ca4424c55933e398ea54974de"
format_stamp: "Formatted at 2026-06-19 16:57:57 on dist-test-slave-xb9n"
server_key: "a411808715154b97b41caa0537cb59e7"
server_key_iv: "cc1da954c0d37ab07124b9e68f5ae2cb"
server_key_version: "encryptionkey@0"
I20260619 16:57:57.780805 19676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:57:57.782095 19691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:57.782256 19676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 16:57:57.782357 19676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "bb732b4ca4424c55933e398ea54974de"
format_stamp: "Formatted at 2026-06-19 16:57:57 on dist-test-slave-xb9n"
server_key: "a411808715154b97b41caa0537cb59e7"
server_key_iv: "cc1da954c0d37ab07124b9e68f5ae2cb"
server_key_version: "encryptionkey@0"
I20260619 16:57:57.782428 19676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:57.802663 19676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:57.805387 19676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:57.805521 19676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:57.810364 19676 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40827
I20260619 16:57:57.810415 19743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40827 every 8 connection(s)
I20260619 16:57:57.810878 19676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:57.811945 19744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:57.815560 19744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de: Bootstrap starting.
I20260619 16:57:57.816352 19744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:57.816699 19744 log.cc:824] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:57.817327 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 19676
I20260619 16:57:57.817479 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:57.817596 19744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de: No bootstrap required, opened a new log
I20260619 16:57:57.817713 31314 external_mini_cluster.cc:1428] Setting key 8e3baaad3f3f61bd9e36802f1de173cd
I20260619 16:57:57.820555 19744 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb732b4ca4424c55933e398ea54974de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40827 } }
I20260619 16:57:57.820749 19744 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:57.820809 19744 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb732b4ca4424c55933e398ea54974de, State: Initialized, Role: FOLLOWER
I20260619 16:57:57.820982 19744 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb732b4ca4424c55933e398ea54974de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40827 } }
I20260619 16:57:57.821074 19744 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:57.821118 19744 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:57.821177 19744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:57.822008 19744 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb732b4ca4424c55933e398ea54974de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40827 } }
I20260619 16:57:57.822131 19744 leader_election.cc:302] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb732b4ca4424c55933e398ea54974de; no voters:
I20260619 16:57:57.822297 19744 leader_election.cc:288] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:57.822335 19749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:57.822450 19749 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [term 1 LEADER]: Becoming Leader. State: Replica: bb732b4ca4424c55933e398ea54974de, State: Running, Role: LEADER
I20260619 16:57:57.822551 19744 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:57.822536 19749 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb732b4ca4424c55933e398ea54974de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40827 } }
I20260619 16:57:57.822860 19749 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb732b4ca4424c55933e398ea54974de. Latest consensus state: current_term: 1 leader_uuid: "bb732b4ca4424c55933e398ea54974de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb732b4ca4424c55933e398ea54974de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40827 } } }
I20260619 16:57:57.822939 19749 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:57.822871 19750 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb732b4ca4424c55933e398ea54974de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb732b4ca4424c55933e398ea54974de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40827 } } }
I20260619 16:57:57.823104 19750 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de [sys.catalog]: This master's current role is: LEADER
W20260619 16:57:57.824376 19764 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:57:57.824431 19764 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:57:57.824658 19754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:57.824875 19754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:57.826483 19754 catalog_manager.cc:1347] Generated new cluster ID: fb4271606e224f1cbe8c22a00c1b4b52
I20260619 16:57:57.826537 19754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:57:57 dist-test-slave-xb9n krb5kdc[19663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 16:57:57.839071 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:57.818178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:38731 (local address 127.0.0.1:40827)
0619 16:57:57.818351 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:57.818359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:57.818394 (+ 35us) server_negotiation.cc:407] Connection header received
0619 16:57:57.819298 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:57.819307 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:57.819370 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:57.819529 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:57.820137 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:57.822028 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:57.823063 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:57.823850 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:57.824037 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:57.835746 (+ 11709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:57.835750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:57.835750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:57.835762 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:57.837486 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:57.837933 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:57.837936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:57.837936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:57.838042 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:57.838271 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:57.838272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:57.838272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:57.838500 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:57.838652 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:57.838791 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 16:57:57.838888 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260619 16:57:57.839986 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:57:57.847311 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:57.841268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:52703 (local address 127.0.0.1:40827)
0619 16:57:57.841338 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:57.841339 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:57.841472 (+ 133us) server_negotiation.cc:407] Connection header received
0619 16:57:57.841540 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:57.841540 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:57.841567 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:57.841621 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:57.842109 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:57.842587 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:57.843282 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:57.843507 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:57.843581 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:57.844719 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:57.844720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:57.844720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:57:57.844727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:57.846150 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:57.846510 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:57.846511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:57.846512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:57.846563 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:57:57.846797 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:57:57.846798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:57:57.846798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:57:57.846906 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:57.847024 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:57.847136 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 16:57:57.847196 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 16:57:57.852758 19754 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:57.853441 19754 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:57.859453 19754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb732b4ca4424c55933e398ea54974de: Generated new TSK 0
I20260619 16:57:57.859637 19754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:57:57.865756 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 19676
W20260619 16:57:57.870260 19661 connection.cc:535] client connection to 127.0.0.1:40827 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40827): Connection reset by peer (error 104)
2026-06-19T16:57:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:57:57 dist-test-slave-xb9n krb5kdc[19775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:57:57 dist-test-slave-xb9n krb5kdc[19775](info): set up 2 sockets
Jun 19 16:57:57 dist-test-slave-xb9n krb5kdc[19775](info): commencing operation
krb5kdc: starting...
W20260619 16:57:58.897421 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:57:58 dist-test-slave-xb9n krb5kdc[19775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:57:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:57:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 16:57:59.008473 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:57:59.086503 19788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:57:59.086686 19788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:57:59.086709 19788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:57:59.087759 19788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:57:59.087791 19788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:57:59.087805 19788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:57:59.087817 19788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:57:59.087828 19788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:57:59.089186 19788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:57:59.089419 19788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:57:59.089599 19788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:57:59.093927 19794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:59.093964 19796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:57:59.093961 19793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:57:59.093972 19788 server_base.cc:1034] running on GCE node
I20260619 16:57:59.094355 19788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:57:59.094631 19788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:57:59.095783 19788 hybrid_clock.cc:648] HybridClock initialized: now 1781888279095772 us; error 25 us; skew 500 ppm
Jun 19 16:57:59 dist-test-slave-xb9n krb5kdc[19775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:57:59.102747 19788 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 16:57:59.103206 19788 webserver.cc:460] Webserver started at http://127.0.0.1:33627/ using document root <none> and password file <none>
I20260619 16:57:59.103411 19788 fs_manager.cc:359] Metadata directory not provided
I20260619 16:57:59.103477 19788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:57:59.103616 19788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:57:59.104579 19788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "b0ccb68f91af4ffc8a5d69f4c1f16211"
format_stamp: "Formatted at 2026-06-19 16:57:59 on dist-test-slave-xb9n"
server_key: "9cc675ab2c592ce2fe293383024afb9f"
server_key_iv: "48b50846d8c9c4b703dbb0a21af0e8c3"
server_key_version: "encryptionkey@0"
I20260619 16:57:59.104938 19788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "b0ccb68f91af4ffc8a5d69f4c1f16211"
format_stamp: "Formatted at 2026-06-19 16:57:59 on dist-test-slave-xb9n"
server_key: "9cc675ab2c592ce2fe293383024afb9f"
server_key_iv: "48b50846d8c9c4b703dbb0a21af0e8c3"
server_key_version: "encryptionkey@0"
I20260619 16:57:59.106341 19788 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:57:59.107097 19803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:57:59.107278 19788 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:57:59.107378 19788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "b0ccb68f91af4ffc8a5d69f4c1f16211"
format_stamp: "Formatted at 2026-06-19 16:57:59 on dist-test-slave-xb9n"
server_key: "9cc675ab2c592ce2fe293383024afb9f"
server_key_iv: "48b50846d8c9c4b703dbb0a21af0e8c3"
server_key_version: "encryptionkey@0"
I20260619 16:57:59.107461 19788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:57:59.132503 19788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:57:59.134348 19788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:57:59.134516 19788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:57:59.138386 19788 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40745
I20260619 16:57:59.138419 19855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40745 every 8 connection(s)
I20260619 16:57:59.138741 19788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:57:59.139540 19856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:57:59.141845 19856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211: Bootstrap starting.
I20260619 16:57:59.142334 19856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211: Neither blocks nor log segments found. Creating new log.
I20260619 16:57:59.142551 19856 log.cc:824] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211: Log is configured to *not* fsync() on all Append() calls
I20260619 16:57:59.142871 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 19788
I20260619 16:57:59.142985 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:57:59.143234 31314 external_mini_cluster.cc:1428] Setting key b6ec5f81067306c8d40319a92860d1b5
I20260619 16:57:59.143390 19856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211: No bootstrap required, opened a new log
I20260619 16:57:59.145978 19856 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ccb68f91af4ffc8a5d69f4c1f16211" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260619 16:57:59.146147 19856 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:57:59.146186 19856 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ccb68f91af4ffc8a5d69f4c1f16211, State: Initialized, Role: FOLLOWER
I20260619 16:57:59.146306 19856 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ccb68f91af4ffc8a5d69f4c1f16211" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260619 16:57:59.146377 19856 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:57:59.146405 19856 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:57:59.146449 19856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:57:59.147132 19856 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ccb68f91af4ffc8a5d69f4c1f16211" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260619 16:57:59.147254 19856 leader_election.cc:302] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0ccb68f91af4ffc8a5d69f4c1f16211; no voters:
I20260619 16:57:59.147421 19856 leader_election.cc:288] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:57:59.147444 19861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:57:59.147547 19861 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [term 1 LEADER]: Becoming Leader. State: Replica: b0ccb68f91af4ffc8a5d69f4c1f16211, State: Running, Role: LEADER
I20260619 16:57:59.147648 19861 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ccb68f91af4ffc8a5d69f4c1f16211" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260619 16:57:59.147676 19856 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:57:59.148170 19861 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0ccb68f91af4ffc8a5d69f4c1f16211" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ccb68f91af4ffc8a5d69f4c1f16211" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } } }
I20260619 16:57:59.148317 19861 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:59.148504 19861 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0ccb68f91af4ffc8a5d69f4c1f16211. Latest consensus state: current_term: 1 leader_uuid: "b0ccb68f91af4ffc8a5d69f4c1f16211" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ccb68f91af4ffc8a5d69f4c1f16211" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } } }
I20260619 16:57:59.148574 19861 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211 [sys.catalog]: This master's current role is: LEADER
I20260619 16:57:59.148802 19870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:57:59.149719 19870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:57:59.150697 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:59.143759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:34753 (local address 127.0.0.1:40745)
0619 16:57:59.143929 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:59.143937 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:59.143970 (+ 33us) server_negotiation.cc:407] Connection header received
0619 16:57:59.144710 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:59.144717 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:59.144764 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:59.144871 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:59.145115 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:59.146005 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:59.147114 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:59.147731 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:59.147866 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:59.148554 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:59.148559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:59.148559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:59.148773 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:59.148813 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:59.148847 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:59.148930 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:59.149143 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 16:57:59.149167 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260619 16:57:59.151331 19870 catalog_manager.cc:1347] Generated new cluster ID: 5acb53942ee34aa08bc829acdbc3f3c6
I20260619 16:57:59.151383 19870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:57:59.154774 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:57:59.159572 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:57:59.156332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:44395 (local address 127.0.0.1:40745)
0619 16:57:59.156373 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 16:57:59.156375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:57:59.156439 (+ 64us) server_negotiation.cc:407] Connection header received
0619 16:57:59.156470 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:57:59.156471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:57:59.156481 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:57:59.156547 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:57:59.156658 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:59.157383 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:59.158326 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:57:59.158631 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:57:59.158705 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:57:59.159333 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:57:59.159334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:57:59.159334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:57:59.159338 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:57:59.159353 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:57:59.159357 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:57:59.159421 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 16:57:59.159491 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 16:57:59.159501 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:57:59.168466 19870 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:57:59.169098 19870 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:57:59.171540 19870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0ccb68f91af4ffc8a5d69f4c1f16211: Generated new TSK 0
I20260619 16:57:59.171712 19870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:57:59.173974 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 19788
W20260619 16:57:59.178100 19773 connection.cc:535] client connection to 127.0.0.1:40745 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40745): Connection reset by peer (error 104)
2026-06-19T16:57:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:57:59 dist-test-slave-xb9n krb5kdc[19887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:57:59 dist-test-slave-xb9n krb5kdc[19887](info): set up 2 sockets
Jun 19 16:57:59 dist-test-slave-xb9n krb5kdc[19887](info): commencing operation
krb5kdc: starting...
W20260619 16:58:00.201684 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:58:00 dist-test-slave-xb9n krb5kdc[19887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 16:58:00.315595 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:00.394241 19900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:00.394438 19900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:00.394461 19900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:00.395541 19900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:00.395576 19900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:00.395589 19900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:00.395602 19900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:00.395614 19900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:00.396977 19900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:00.397156 19900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:00.397317 19900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:00.401947 19905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:00.401984 19900 server_base.cc:1034] running on GCE node
W20260619 16:58:00.401926 19906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:00.401937 19908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:00.402449 19900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:00.402750 19900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:00.403909 19900 hybrid_clock.cc:648] HybridClock initialized: now 1781888280403901 us; error 35 us; skew 500 ppm
Jun 19 16:58:00 dist-test-slave-xb9n krb5kdc[19887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888280, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:00.406419 19900 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 16:58:00.406889 19900 webserver.cc:460] Webserver started at http://127.0.0.1:40219/ using document root <none> and password file <none>
I20260619 16:58:00.407131 19900 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:00.407200 19900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:00.407348 19900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:00.408394 19900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "82e1ad4dd9944f15976509ca9d7b0390"
format_stamp: "Formatted at 2026-06-19 16:58:00 on dist-test-slave-xb9n"
server_key: "9a62549a126515e81b7d88de237f2d4b"
server_key_iv: "9f4293bd20a630c885ea67c8d1e47da3"
server_key_version: "encryptionkey@0"
I20260619 16:58:00.408764 19900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "82e1ad4dd9944f15976509ca9d7b0390"
format_stamp: "Formatted at 2026-06-19 16:58:00 on dist-test-slave-xb9n"
server_key: "9a62549a126515e81b7d88de237f2d4b"
server_key_iv: "9f4293bd20a630c885ea67c8d1e47da3"
server_key_version: "encryptionkey@0"
I20260619 16:58:00.410184 19900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 16:58:00.410979 19915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:00.411172 19900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:00.411278 19900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "82e1ad4dd9944f15976509ca9d7b0390"
format_stamp: "Formatted at 2026-06-19 16:58:00 on dist-test-slave-xb9n"
server_key: "9a62549a126515e81b7d88de237f2d4b"
server_key_iv: "9f4293bd20a630c885ea67c8d1e47da3"
server_key_version: "encryptionkey@0"
I20260619 16:58:00.411350 19900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:00.414633 19900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:00.416950 19900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:00.417101 19900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:00.421005 19900 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41117
I20260619 16:58:00.421033 19967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41117 every 8 connection(s)
I20260619 16:58:00.421343 19900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:00.422314 19968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:00.424379 19968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390: Bootstrap starting.
I20260619 16:58:00.424903 19968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:00.425190 19968 log.cc:824] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:00.425855 19968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390: No bootstrap required, opened a new log
I20260619 16:58:00.427363 19968 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e1ad4dd9944f15976509ca9d7b0390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41117 } }
I20260619 16:58:00.427489 19968 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:00.427511 19968 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82e1ad4dd9944f15976509ca9d7b0390, State: Initialized, Role: FOLLOWER
I20260619 16:58:00.427654 19968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e1ad4dd9944f15976509ca9d7b0390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41117 } }
I20260619 16:58:00.427734 19968 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:00.427775 19968 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:00.427824 19968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:00.428339 19968 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e1ad4dd9944f15976509ca9d7b0390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41117 } }
I20260619 16:58:00.428421 19968 leader_election.cc:302] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82e1ad4dd9944f15976509ca9d7b0390; no voters:
I20260619 16:58:00.428586 19968 leader_election.cc:288] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:00.428661 19971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:00.428820 19971 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [term 1 LEADER]: Becoming Leader. State: Replica: 82e1ad4dd9944f15976509ca9d7b0390, State: Running, Role: LEADER
I20260619 16:58:00.428830 19968 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:00.428918 19971 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e1ad4dd9944f15976509ca9d7b0390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41117 } }
I20260619 16:58:00.429334 19973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82e1ad4dd9944f15976509ca9d7b0390. Latest consensus state: current_term: 1 leader_uuid: "82e1ad4dd9944f15976509ca9d7b0390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e1ad4dd9944f15976509ca9d7b0390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41117 } } }
I20260619 16:58:00.429432 19973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:00.429311 19972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82e1ad4dd9944f15976509ca9d7b0390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e1ad4dd9944f15976509ca9d7b0390" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41117 } } }
I20260619 16:58:00.429553 19972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:00.429628 19975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:00.429697 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 19900
I20260619 16:58:00.429819 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:58:00.429997 31314 external_mini_cluster.cc:1428] Setting key b0487eb0384f3fc23157a2f409550761
I20260619 16:58:00.430104 19975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:00.431533 19975 catalog_manager.cc:1347] Generated new cluster ID: 17d1d9b0ba28490f99be02c8e8791dad
I20260619 16:58:00.431584 19975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:00.432766 19984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:00.430617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:35341 (local address 127.0.0.1:41117)
0619 16:58:00.430760 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:00.430769 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:00.430920 (+ 151us) server_negotiation.cc:407] Connection header received
0619 16:58:00.431130 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:00.431138 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:00.431173 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:00.431376 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:00.432355 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:00.432359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:00.432359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:00.432363 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:00.432403 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:00.432417 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:00.432493 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:00.432558 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 16:58:00.432599 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260619 16:58:00.433763 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:00.436292 19984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:00.435092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:47545 (local address 127.0.0.1:41117)
0619 16:58:00.435171 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:00.435173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:00.435279 (+ 106us) server_negotiation.cc:407] Connection header received
0619 16:58:00.435369 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:00.435370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:00.435374 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:00.435403 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:00.436051 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:00.436052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:00.436052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:00.436056 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:00.436068 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:00.436072 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:00.436103 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:00.436220 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 16:58:00.436235 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 16:58:00.441665 19975 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:00.441843 19975 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:00.450484 19975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82e1ad4dd9944f15976509ca9d7b0390: Generated new TSK 0
I20260619 16:58:00.450645 19975 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:00.457403 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 19900
2026-06-19T16:58:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:00 dist-test-slave-xb9n krb5kdc[19999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:00 dist-test-slave-xb9n krb5kdc[19999](info): set up 2 sockets
Jun 19 16:58:00 dist-test-slave-xb9n krb5kdc[19999](info): commencing operation
krb5kdc: starting...
W20260619 16:58:01.489504 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:58:01 dist-test-slave-xb9n krb5kdc[19999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 16:58:01.601753 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40747
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:01.681771 20012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:01.681982 20012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:01.682004 20012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:01.683141 20012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:01.683178 20012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:01.683193 20012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:01.683205 20012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:01.683218 20012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:01.684646 20012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40747
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:01.684834 20012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:01.685000 20012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:58:01.689682 20012 server_base.cc:1034] running on GCE node
W20260619 16:58:01.689637 20017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:01.689637 20020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:01.689631 20018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:01.690092 20012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:01.690433 20012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:01.691598 20012 hybrid_clock.cc:648] HybridClock initialized: now 1781888281691591 us; error 33 us; skew 500 ppm
Jun 19 16:58:01 dist-test-slave-xb9n krb5kdc[19999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888281, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:01.693996 20012 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 16:58:01.694402 20012 webserver.cc:460] Webserver started at http://127.0.0.1:41411/ using document root <none> and password file <none>
I20260619 16:58:01.694629 20012 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:01.694695 20012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:01.694845 20012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:01.695875 20012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "558dd49a9ad4424d95c62ea06f74f8ea"
format_stamp: "Formatted at 2026-06-19 16:58:01 on dist-test-slave-xb9n"
server_key: "ce781d40c7f76fd105f00760ff5b35b8"
server_key_iv: "3caf47264d7b80de4765d1baed528c9f"
server_key_version: "encryptionkey@0"
I20260619 16:58:01.696233 20012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "558dd49a9ad4424d95c62ea06f74f8ea"
format_stamp: "Formatted at 2026-06-19 16:58:01 on dist-test-slave-xb9n"
server_key: "ce781d40c7f76fd105f00760ff5b35b8"
server_key_iv: "3caf47264d7b80de4765d1baed528c9f"
server_key_version: "encryptionkey@0"
I20260619 16:58:01.697531 20012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 16:58:01.698216 20027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:01.698370 20012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:01.698462 20012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "558dd49a9ad4424d95c62ea06f74f8ea"
format_stamp: "Formatted at 2026-06-19 16:58:01 on dist-test-slave-xb9n"
server_key: "ce781d40c7f76fd105f00760ff5b35b8"
server_key_iv: "3caf47264d7b80de4765d1baed528c9f"
server_key_version: "encryptionkey@0"
I20260619 16:58:01.698544 20012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:01.725023 20012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:01.727007 20012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:01.727164 20012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:01.731014 20012 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40747
I20260619 16:58:01.731050 20079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40747 every 8 connection(s)
I20260619 16:58:01.731371 20012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:01.732175 20080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:01.734406 20080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea: Bootstrap starting.
I20260619 16:58:01.734943 20080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:01.735222 20080 log.cc:824] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:01.735875 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 20012
I20260619 16:58:01.735901 20080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea: No bootstrap required, opened a new log
I20260619 16:58:01.736002 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:58:01.736176 31314 external_mini_cluster.cc:1428] Setting key e452376aeddd45fb2fda2d4ad5711f92
I20260619 16:58:01.737874 20080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558dd49a9ad4424d95c62ea06f74f8ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40747 } }
I20260619 16:58:01.738094 20080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:01.738170 20080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558dd49a9ad4424d95c62ea06f74f8ea, State: Initialized, Role: FOLLOWER
I20260619 16:58:01.738307 20080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558dd49a9ad4424d95c62ea06f74f8ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40747 } }
I20260619 16:58:01.738391 20080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:01.738423 20080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:01.738478 20080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:01.739398 20080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558dd49a9ad4424d95c62ea06f74f8ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40747 } }
I20260619 16:58:01.739518 20080 leader_election.cc:302] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 558dd49a9ad4424d95c62ea06f74f8ea; no voters:
I20260619 16:58:01.739702 20080 leader_election.cc:288] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:01.739718 20085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:01.739881 20085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [term 1 LEADER]: Becoming Leader. State: Replica: 558dd49a9ad4424d95c62ea06f74f8ea, State: Running, Role: LEADER
I20260619 16:58:01.739995 20080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:01.740027 20085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558dd49a9ad4424d95c62ea06f74f8ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40747 } }
I20260619 16:58:01.740430 20085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 558dd49a9ad4424d95c62ea06f74f8ea. Latest consensus state: current_term: 1 leader_uuid: "558dd49a9ad4424d95c62ea06f74f8ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558dd49a9ad4424d95c62ea06f74f8ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40747 } } }
I20260619 16:58:01.740674 20085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:01.740590 20086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "558dd49a9ad4424d95c62ea06f74f8ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558dd49a9ad4424d95c62ea06f74f8ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40747 } } }
I20260619 16:58:01.741035 20086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:01.741211 20094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:01.741761 20094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 16:58:01 dist-test-slave-xb9n krb5kdc[19999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 16:58:01.744777 20094 catalog_manager.cc:1347] Generated new cluster ID: d83f4b9b84b24ee0a6f6ed993d1ea05a
I20260619 16:58:01.744830 20094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:01.747547 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:01.736734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:59023 (local address 127.0.0.1:40747)
0619 16:58:01.736904 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:01.736911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:01.736935 (+ 24us) server_negotiation.cc:407] Connection header received
0619 16:58:01.737855 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:01.737861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:01.737907 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:01.738016 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:01.738813 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:01.739811 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:01.740815 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:01.741428 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:01.741525 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:01.744215 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:01.744219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:01.744220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:01.744230 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:01.746016 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:01.746410 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:01.746413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:01.746413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:01.746524 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:01.746758 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:01.746758 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:01.746759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:01.747002 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:01.747173 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:01.747292 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 16:58:01.747390 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260619 16:58:01.748656 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:01.757062 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:01.750132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:51041 (local address 127.0.0.1:40747)
0619 16:58:01.750165 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:01.750167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:01.750293 (+ 126us) server_negotiation.cc:407] Connection header received
0619 16:58:01.750350 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:01.750351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:01.750360 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:01.750407 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:01.751044 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:01.751741 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:01.752782 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:01.753064 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:01.753118 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:01.754405 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:01.754406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:01.754407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:01.754412 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:01.755859 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:01.756280 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:01.756281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:01.756281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:01.756333 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:01.756563 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:01.756564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:01.756564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:01.756679 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:01.756764 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:01.756921 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 16:58:01.756979 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:58:01.758544 20094 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:01.759197 20094 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:01.764569 20094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 558dd49a9ad4424d95c62ea06f74f8ea: Generated new TSK 0
I20260619 16:58:01.764739 20094 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:01.766788 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 20012
W20260619 16:58:01.770846 19997 connection.cc:535] client connection to 127.0.0.1:40747 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40747): Connection reset by peer (error 104)
2026-06-19T16:58:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:01 dist-test-slave-xb9n krb5kdc[20111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:01 dist-test-slave-xb9n krb5kdc[20111](info): set up 2 sockets
Jun 19 16:58:01 dist-test-slave-xb9n krb5kdc[20111](info): commencing operation
krb5kdc: starting...
W20260619 16:58:02.805565 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:58:02 dist-test-slave-xb9n krb5kdc[20111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 16:58:02.918222 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:02.997496 20124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:02.997721 20124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:02.997756 20124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:02.998847 20124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:02.998915 20124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:02.998939 20124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:02.998967 20124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:02.998998 20124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:03.000419 20124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:03.000702 20124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:03.000927 20124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:03.006135 20132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:03.006208 20124 server_base.cc:1034] running on GCE node
W20260619 16:58:03.006115 20130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:03.006115 20129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:03.006671 20124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:03.006953 20124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:03.008090 20124 hybrid_clock.cc:648] HybridClock initialized: now 1781888283008076 us; error 36 us; skew 500 ppm
Jun 19 16:58:03 dist-test-slave-xb9n krb5kdc[20111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888283, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:03.010555 20124 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 16:58:03.011018 20124 webserver.cc:460] Webserver started at http://127.0.0.1:36069/ using document root <none> and password file <none>
I20260619 16:58:03.011260 20124 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:03.011328 20124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:03.011459 20124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:03.012471 20124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "fa9ff37f65a34f0ebd729bb84eb81421"
format_stamp: "Formatted at 2026-06-19 16:58:03 on dist-test-slave-xb9n"
server_key: "62d2b12b5cc90cb3d407ce2217fd92ec"
server_key_iv: "e93f81d71cf663dcd4500658d16874fc"
server_key_version: "encryptionkey@0"
I20260619 16:58:03.012828 20124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "fa9ff37f65a34f0ebd729bb84eb81421"
format_stamp: "Formatted at 2026-06-19 16:58:03 on dist-test-slave-xb9n"
server_key: "62d2b12b5cc90cb3d407ce2217fd92ec"
server_key_iv: "e93f81d71cf663dcd4500658d16874fc"
server_key_version: "encryptionkey@0"
I20260619 16:58:03.014217 20124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:03.014922 20139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:03.015093 20124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:03.015190 20124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "fa9ff37f65a34f0ebd729bb84eb81421"
format_stamp: "Formatted at 2026-06-19 16:58:03 on dist-test-slave-xb9n"
server_key: "62d2b12b5cc90cb3d407ce2217fd92ec"
server_key_iv: "e93f81d71cf663dcd4500658d16874fc"
server_key_version: "encryptionkey@0"
I20260619 16:58:03.015280 20124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:03.030359 20124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:03.032061 20124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:03.032217 20124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:03.036128 20124 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33163
I20260619 16:58:03.036166 20191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33163 every 8 connection(s)
I20260619 16:58:03.036507 20124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:03.037344 20192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:03.039536 20192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421: Bootstrap starting.
I20260619 16:58:03.040095 20192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:03.040381 20192 log.cc:824] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:03.041023 20192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421: No bootstrap required, opened a new log
I20260619 16:58:03.042523 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 20124
I20260619 16:58:03.042649 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:58:03.042857 31314 external_mini_cluster.cc:1428] Setting key 48f89b0176e32699fe2de4083dd7b8c6
I20260619 16:58:03.042795 20192 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9ff37f65a34f0ebd729bb84eb81421" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } }
I20260619 16:58:03.042961 20192 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:03.043008 20192 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9ff37f65a34f0ebd729bb84eb81421, State: Initialized, Role: FOLLOWER
I20260619 16:58:03.043202 20192 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9ff37f65a34f0ebd729bb84eb81421" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } }
I20260619 16:58:03.043323 20192 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:03.043375 20192 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:03.043462 20192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:03.044447 20192 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9ff37f65a34f0ebd729bb84eb81421" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } }
I20260619 16:58:03.044531 20192 leader_election.cc:302] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa9ff37f65a34f0ebd729bb84eb81421; no voters:
I20260619 16:58:03.044726 20192 leader_election.cc:288] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:03.044746 20197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:03.044854 20197 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [term 1 LEADER]: Becoming Leader. State: Replica: fa9ff37f65a34f0ebd729bb84eb81421, State: Running, Role: LEADER
I20260619 16:58:03.044975 20192 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:03.044982 20197 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9ff37f65a34f0ebd729bb84eb81421" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } }
I20260619 16:58:03.046645 20198 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa9ff37f65a34f0ebd729bb84eb81421" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9ff37f65a34f0ebd729bb84eb81421" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } } }
W20260619 16:58:03.046733 20212 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:58:03.046871 20212 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:58:03.046916 20199 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa9ff37f65a34f0ebd729bb84eb81421. Latest consensus state: current_term: 1 leader_uuid: "fa9ff37f65a34f0ebd729bb84eb81421" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9ff37f65a34f0ebd729bb84eb81421" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33163 } } }
I20260619 16:58:03.047000 20199 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:03.046797 20198 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:03.047508 20214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:03.047725 20214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:03.048919 20196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:03.043332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:49203 (local address 127.0.0.1:33163)
0619 16:58:03.043501 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:03.043510 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:03.043565 (+ 55us) server_negotiation.cc:407] Connection header received
0619 16:58:03.043676 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:03.043681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:03.043746 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:03.043845 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:03.044118 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:03.045237 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:03.046438 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:03.047032 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:03.047135 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:03.047902 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:03.047905 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:03.047906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:03.048075 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:03.048102 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:03.048126 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:03.048240 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:03.048682 (+ 442us) server_negotiation.cc:299] Negotiation successful
0619 16:58:03.048709 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260619 16:58:03.049288 20214 catalog_manager.cc:1347] Generated new cluster ID: 90341fae0f514332831de5849e0a3d5a
I20260619 16:58:03.049340 20214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:03.049765 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:03.053840 20196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:03.051073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:52809 (local address 127.0.0.1:33163)
0619 16:58:03.051108 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:03.051110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:03.051221 (+ 111us) server_negotiation.cc:407] Connection header received
0619 16:58:03.051283 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:03.051284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:03.051294 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:03.051367 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:03.051509 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:03.051954 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:03.052655 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:03.052870 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:03.052940 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:03.053602 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:03.053604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:03.053604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:03.053609 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:03.053621 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:03.053625 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:03.053678 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:03.053737 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 16:58:03.053748 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:58:03.070606 20214 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:03.071348 20214 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:03.075244 20214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa9ff37f65a34f0ebd729bb84eb81421: Generated new TSK 0
I20260619 16:58:03.075405 20214 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:03.086796 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 20124
W20260619 16:58:03.090937 20109 connection.cc:535] client connection to 127.0.0.1:33163 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33163): Connection reset by peer (error 104)
2026-06-19T16:58:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:03 dist-test-slave-xb9n krb5kdc[20223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:03 dist-test-slave-xb9n krb5kdc[20223](info): set up 2 sockets
Jun 19 16:58:03 dist-test-slave-xb9n krb5kdc[20223](info): commencing operation
krb5kdc: starting...
W20260619 16:58:04.109603 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:58:04 dist-test-slave-xb9n krb5kdc[20223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 16:58:04.224255 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37189
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:04.303143 20236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:04.303380 20236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:04.303419 20236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:04.304543 20236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:04.304615 20236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:04.304639 20236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:04.304672 20236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:04.304702 20236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:04.306232 20236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37189
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:04.306495 20236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:04.306706 20236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:04.311534 20241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:04.311592 20236 server_base.cc:1034] running on GCE node
W20260619 16:58:04.311563 20242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:04.311563 20244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:04.311966 20236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:04.312247 20236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:04.313414 20236 hybrid_clock.cc:648] HybridClock initialized: now 1781888284313394 us; error 49 us; skew 500 ppm
Jun 19 16:58:04 dist-test-slave-xb9n krb5kdc[20223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888284, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:04.319218 20236 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 16:58:04.319614 20236 webserver.cc:460] Webserver started at http://127.0.0.1:45949/ using document root <none> and password file <none>
I20260619 16:58:04.319852 20236 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:04.319929 20236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:04.320065 20236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:04.321018 20236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "5561b48f14eb47b48541f4b6f11e8ee3"
format_stamp: "Formatted at 2026-06-19 16:58:04 on dist-test-slave-xb9n"
server_key: "e6bc9f874da0db211691c57dba182c5e"
server_key_iv: "366602755fd28412b03882544ba7c126"
server_key_version: "encryptionkey@0"
I20260619 16:58:04.321460 20236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "5561b48f14eb47b48541f4b6f11e8ee3"
format_stamp: "Formatted at 2026-06-19 16:58:04 on dist-test-slave-xb9n"
server_key: "e6bc9f874da0db211691c57dba182c5e"
server_key_iv: "366602755fd28412b03882544ba7c126"
server_key_version: "encryptionkey@0"
I20260619 16:58:04.322738 20236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:04.323480 20251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:04.323673 20236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:04.323766 20236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "5561b48f14eb47b48541f4b6f11e8ee3"
format_stamp: "Formatted at 2026-06-19 16:58:04 on dist-test-slave-xb9n"
server_key: "e6bc9f874da0db211691c57dba182c5e"
server_key_iv: "366602755fd28412b03882544ba7c126"
server_key_version: "encryptionkey@0"
I20260619 16:58:04.323853 20236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:04.327054 20236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:04.329221 20236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:04.329432 20236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:04.333017 20236 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37189
I20260619 16:58:04.333045 20303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37189 every 8 connection(s)
I20260619 16:58:04.333315 20236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:04.334192 20304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:04.336184 20304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3: Bootstrap starting.
I20260619 16:58:04.336652 20304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:04.336907 20304 log.cc:824] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:04.337585 20304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3: No bootstrap required, opened a new log
I20260619 16:58:04.338266 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 20236
I20260619 16:58:04.338382 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:58:04.338548 31314 external_mini_cluster.cc:1428] Setting key cc96b5ad678af10b3cbbef5790320674
I20260619 16:58:04.339624 20304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5561b48f14eb47b48541f4b6f11e8ee3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37189 } }
I20260619 16:58:04.339797 20304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:04.339834 20304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5561b48f14eb47b48541f4b6f11e8ee3, State: Initialized, Role: FOLLOWER
I20260619 16:58:04.340022 20304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5561b48f14eb47b48541f4b6f11e8ee3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37189 } }
I20260619 16:58:04.340116 20304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:04.340168 20304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:04.340216 20304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:04.340828 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:04.338995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:46843 (local address 127.0.0.1:37189)
0619 16:58:04.339221 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:04.339229 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:04.339262 (+ 33us) server_negotiation.cc:407] Connection header received
0619 16:58:04.339586 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:04.339594 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:04.339626 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:04.339770 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:04.340371 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:04.340373 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:04.340374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:04.340377 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:04.340409 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:04.340420 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:04.340465 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:04.340586 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 16:58:04.340634 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
I20260619 16:58:04.341053 20304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5561b48f14eb47b48541f4b6f11e8ee3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37189 } }
I20260619 16:58:04.341151 20304 leader_election.cc:302] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5561b48f14eb47b48541f4b6f11e8ee3; no voters:
I20260619 16:58:04.341290 20304 leader_election.cc:288] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:04.341445 20309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:04.341537 20304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:04.341650 20309 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 5561b48f14eb47b48541f4b6f11e8ee3, State: Running, Role: LEADER
I20260619 16:58:04.341760 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:04.341745 20309 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5561b48f14eb47b48541f4b6f11e8ee3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37189 } }
I20260619 16:58:04.342064 20309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5561b48f14eb47b48541f4b6f11e8ee3. Latest consensus state: current_term: 1 leader_uuid: "5561b48f14eb47b48541f4b6f11e8ee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5561b48f14eb47b48541f4b6f11e8ee3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37189 } } }
I20260619 16:58:04.342073 20310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5561b48f14eb47b48541f4b6f11e8ee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5561b48f14eb47b48541f4b6f11e8ee3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37189 } } }
I20260619 16:58:04.342150 20309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:04.342170 20310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3 [sys.catalog]: This master's current role is: LEADER
W20260619 16:58:04.343328 20326 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:58:04.343384 20326 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:58:04.343567 20318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:04.343743 20318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:04.344493 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:04.343302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:39697 (local address 127.0.0.1:37189)
0619 16:58:04.343334 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:04.343336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:04.343544 (+ 208us) server_negotiation.cc:407] Connection header received
0619 16:58:04.343657 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:04.343658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:04.343663 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:04.343711 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:04.344275 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:04.344276 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:04.344276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:04.344281 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:04.344301 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:04.344305 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:04.344352 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:04.344406 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 16:58:04.344422 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 16:58:04.345129 20318 catalog_manager.cc:1347] Generated new cluster ID: bb987daefb264e7695da8c4f086364ca
I20260619 16:58:04.345178 20318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:04.350858 20318 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:04.351023 20318 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:04.353410 20318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5561b48f14eb47b48541f4b6f11e8ee3: Generated new TSK 0
I20260619 16:58:04.353556 20318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:04.355449 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 20236
2026-06-19T16:58:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1268 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:04 dist-test-slave-xb9n krb5kdc[20335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:04 dist-test-slave-xb9n krb5kdc[20335](info): set up 2 sockets
Jun 19 16:58:04 dist-test-slave-xb9n krb5kdc[20335](info): commencing operation
krb5kdc: starting...
W20260619 16:58:05.393275 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:58:05 dist-test-slave-xb9n krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:58:05.505286 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:43821
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:38103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:43821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:05.584553 20348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:05.584749 20348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:05.584770 20348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:05.585983 20348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:05.586023 20348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:05.586037 20348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:05.586071 20348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:05.586098 20348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:05.587519 20348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:43821
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:43821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:05.587764 20348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:05.587978 20348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:05.592581 20354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:05.592681 20356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:05.592582 20353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:05.592837 20348 server_base.cc:1034] running on GCE node
I20260619 16:58:05.593034 20348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:05.593287 20348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:05.594422 20348 hybrid_clock.cc:648] HybridClock initialized: now 1781888285594415 us; error 29 us; skew 500 ppm
Jun 19 16:58:05 dist-test-slave-xb9n krb5kdc[20335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888285, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:05.596792 20348 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:58:05.597257 20348 webserver.cc:460] Webserver started at http://127.30.148.190:42211/ using document root <none> and password file <none>
I20260619 16:58:05.597546 20348 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:05.597615 20348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:05.597748 20348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:05.598721 20348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "ba5683865593481595f8ab9cc5fb5e56"
format_stamp: "Formatted at 2026-06-19 16:58:05 on dist-test-slave-xb9n"
server_key: "4da798f1bc3472d331c9bef2b07a325e"
server_key_iv: "c082871db17fc2047e965cedec54f4cd"
server_key_version: "encryptionkey@0"
I20260619 16:58:05.599086 20348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "ba5683865593481595f8ab9cc5fb5e56"
format_stamp: "Formatted at 2026-06-19 16:58:05 on dist-test-slave-xb9n"
server_key: "4da798f1bc3472d331c9bef2b07a325e"
server_key_iv: "c082871db17fc2047e965cedec54f4cd"
server_key_version: "encryptionkey@0"
I20260619 16:58:05.600409 20348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:05.601112 20363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:05.601300 20348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:05.601418 20348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "ba5683865593481595f8ab9cc5fb5e56"
format_stamp: "Formatted at 2026-06-19 16:58:05 on dist-test-slave-xb9n"
server_key: "4da798f1bc3472d331c9bef2b07a325e"
server_key_iv: "c082871db17fc2047e965cedec54f4cd"
server_key_version: "encryptionkey@0"
I20260619 16:58:05.601492 20348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:05.619136 20348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:05.620929 20348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:05.621089 20348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:05.624948 20348 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:43821
I20260619 16:58:05.624982 20415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:43821 every 8 connection(s)
I20260619 16:58:05.625319 20348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:05.626173 20416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:05.628365 20416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56: Bootstrap starting.
I20260619 16:58:05.628931 20416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:05.629282 20416 log.cc:824] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:05.629379 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 20348
I20260619 16:58:05.629487 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:58:05.629735 31314 external_mini_cluster.cc:1428] Setting key 678db2db961e58f91be394d89a501874
I20260619 16:58:05.630123 20416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56: No bootstrap required, opened a new log
I20260619 16:58:05.631784 20416 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5683865593481595f8ab9cc5fb5e56" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43821 } }
I20260619 16:58:05.631943 20416 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:05.631991 20416 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba5683865593481595f8ab9cc5fb5e56, State: Initialized, Role: FOLLOWER
I20260619 16:58:05.632125 20416 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5683865593481595f8ab9cc5fb5e56" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43821 } }
I20260619 16:58:05.632227 20416 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:05.632261 20416 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:05.632325 20416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:05.633437 20416 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5683865593481595f8ab9cc5fb5e56" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43821 } }
I20260619 16:58:05.633536 20416 leader_election.cc:302] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba5683865593481595f8ab9cc5fb5e56; no voters:
I20260619 16:58:05.633706 20416 leader_election.cc:288] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:05.633728 20421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:05.633831 20421 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [term 1 LEADER]: Becoming Leader. State: Replica: ba5683865593481595f8ab9cc5fb5e56, State: Running, Role: LEADER
I20260619 16:58:05.633910 20421 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5683865593481595f8ab9cc5fb5e56" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43821 } }
I20260619 16:58:05.633955 20416 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:05.634236 20423 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba5683865593481595f8ab9cc5fb5e56. Latest consensus state: current_term: 1 leader_uuid: "ba5683865593481595f8ab9cc5fb5e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5683865593481595f8ab9cc5fb5e56" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43821 } } }
I20260619 16:58:05.634323 20423 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:05.634497 20422 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba5683865593481595f8ab9cc5fb5e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba5683865593481595f8ab9cc5fb5e56" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 43821 } } }
I20260619 16:58:05.634567 20422 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:05.634549 20425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:05.635264 20425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:05.637286 20425 catalog_manager.cc:1347] Generated new cluster ID: d8a154456374498ab6dd21b2b1e2f114
I20260619 16:58:05.637342 20425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:58:05 dist-test-slave-xb9n krb5kdc[20335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:58:05.641378 20425 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:05.641872 20425 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:05.642730 20420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:05.630275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33139 (local address 127.30.148.190:43821)
0619 16:58:05.630463 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:05.630472 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:05.630508 (+ 36us) server_negotiation.cc:407] Connection header received
0619 16:58:05.631775 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:05.631781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:05.631836 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:05.631947 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:05.632872 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:05.634062 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:05.635184 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:05.635770 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:05.635886 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:05.638663 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:05.638667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:05.638667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:05.638679 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:05.640600 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:05.641085 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:05.641088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:05.641088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:05.641208 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:05.641487 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:05.641488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:05.641488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:05.641772 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:05.642330 (+ 558us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:05.642416 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 16:58:05.642523 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":107,"threads_started":1}
I20260619 16:58:05.643688 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:05.645004 20425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba5683865593481595f8ab9cc5fb5e56: Generated new TSK 0
I20260619 16:58:05.645164 20425 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:05.651485 20420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:05.645111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56167 (local address 127.30.148.190:43821)
0619 16:58:05.645144 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:05.645147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:05.645415 (+ 268us) server_negotiation.cc:407] Connection header received
0619 16:58:05.645474 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:05.645475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:05.645485 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:05.645552 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:05.646010 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:05.646774 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:05.647433 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:05.647748 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:05.647819 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:05.648912 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:05.648914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:05.648914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:05.648921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:05.650400 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:05.650767 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:05.650768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:05.650769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:05.650826 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:05.651026 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:05.651027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:05.651027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:05.651150 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:05.651250 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:05.651339 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 16:58:05.651398 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 16:58:05.653380 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 20348
W20260619 16:58:05.657547 20333 connection.cc:535] client connection to 127.30.148.190:43821 recv error: Network error: failed to read from TLS socket (remote: 127.30.148.190:43821): Connection reset by peer (error 104)
2026-06-19T16:58:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:05 dist-test-slave-xb9n krb5kdc[20447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:05 dist-test-slave-xb9n krb5kdc[20447](info): set up 2 sockets
Jun 19 16:58:05 dist-test-slave-xb9n krb5kdc[20447](info): commencing operation
krb5kdc: starting...
W20260619 16:58:06.681348 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:58:06 dist-test-slave-xb9n krb5kdc[20447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:58:06.791016 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:34901
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:41977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:34901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:06.869354 20460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:06.869556 20460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:06.869578 20460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:06.870765 20460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:06.870807 20460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:06.870820 20460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:06.870854 20460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:06.870882 20460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:06.872406 20460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:41977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:34901
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:34901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:06.872648 20460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:06.872870 20460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:06.877564 20466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:06.877633 20460 server_base.cc:1034] running on GCE node
W20260619 16:58:06.877581 20465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:06.877585 20468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:06.878057 20460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:06.878331 20460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:06.879470 20460 hybrid_clock.cc:648] HybridClock initialized: now 1781888286879458 us; error 29 us; skew 500 ppm
Jun 19 16:58:06 dist-test-slave-xb9n krb5kdc[20447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888286, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:06.881814 20460 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:58:06.882226 20460 webserver.cc:460] Webserver started at http://127.30.148.190:40379/ using document root <none> and password file <none>
I20260619 16:58:06.882460 20460 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:06.882526 20460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:06.882671 20460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:06.883657 20460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "66260414de7f41039c67ca7020e46b39"
format_stamp: "Formatted at 2026-06-19 16:58:06 on dist-test-slave-xb9n"
server_key: "ba67144e0ce1f9bd10153dacf87b5d8e"
server_key_iv: "89ea9f8d2f2cffbd3b6f58af93bb76cb"
server_key_version: "encryptionkey@0"
I20260619 16:58:06.883998 20460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "66260414de7f41039c67ca7020e46b39"
format_stamp: "Formatted at 2026-06-19 16:58:06 on dist-test-slave-xb9n"
server_key: "ba67144e0ce1f9bd10153dacf87b5d8e"
server_key_iv: "89ea9f8d2f2cffbd3b6f58af93bb76cb"
server_key_version: "encryptionkey@0"
I20260619 16:58:06.885260 20460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 16:58:06.885958 20475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:06.886113 20460 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:06.886206 20460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "66260414de7f41039c67ca7020e46b39"
format_stamp: "Formatted at 2026-06-19 16:58:06 on dist-test-slave-xb9n"
server_key: "ba67144e0ce1f9bd10153dacf87b5d8e"
server_key_iv: "89ea9f8d2f2cffbd3b6f58af93bb76cb"
server_key_version: "encryptionkey@0"
I20260619 16:58:06.886284 20460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:06.906126 20460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:06.907797 20460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:06.907953 20460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:06.911775 20460 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:34901
I20260619 16:58:06.911808 20527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:34901 every 8 connection(s)
I20260619 16:58:06.912156 20460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:06.912978 20528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:06.915136 20528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39: Bootstrap starting.
I20260619 16:58:06.915138 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 20460
I20260619 16:58:06.915263 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:58:06.915455 31314 external_mini_cluster.cc:1428] Setting key 904d3e6426cbd3973a3f1786d25177a4
I20260619 16:58:06.915795 20528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:06.916142 20528 log.cc:824] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:06.916792 20528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39: No bootstrap required, opened a new log
I20260619 16:58:06.918641 20528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66260414de7f41039c67ca7020e46b39" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 34901 } }
I20260619 16:58:06.918788 20528 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:06.918833 20528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66260414de7f41039c67ca7020e46b39, State: Initialized, Role: FOLLOWER
I20260619 16:58:06.918991 20528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66260414de7f41039c67ca7020e46b39" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 34901 } }
I20260619 16:58:06.919080 20528 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:06.919140 20528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:06.919194 20528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:06.919694 20528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66260414de7f41039c67ca7020e46b39" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 34901 } }
I20260619 16:58:06.919821 20528 leader_election.cc:302] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66260414de7f41039c67ca7020e46b39; no voters:
I20260619 16:58:06.919986 20528 leader_election.cc:288] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:06.920015 20533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:06.920234 20528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:06.920361 20533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [term 1 LEADER]: Becoming Leader. State: Replica: 66260414de7f41039c67ca7020e46b39, State: Running, Role: LEADER
I20260619 16:58:06.920457 20533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66260414de7f41039c67ca7020e46b39" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 34901 } }
I20260619 16:58:06.920917 20533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66260414de7f41039c67ca7020e46b39. Latest consensus state: current_term: 1 leader_uuid: "66260414de7f41039c67ca7020e46b39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66260414de7f41039c67ca7020e46b39" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 34901 } } }
I20260619 16:58:06.921010 20533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:06.920930 20534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66260414de7f41039c67ca7020e46b39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66260414de7f41039c67ca7020e46b39" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 34901 } } }
I20260619 16:58:06.921100 20534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:06.921209 20542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:06.921335 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:06.915944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60865 (local address 127.30.148.190:34901)
0619 16:58:06.916151 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:06.916159 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:06.916213 (+ 54us) server_negotiation.cc:407] Connection header received
0619 16:58:06.916645 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:06.916651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:06.916696 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:06.916838 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:06.917050 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:06.917988 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:06.918786 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:06.919378 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:06.919492 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:06.920225 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:06.920229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:06.920229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:06.920472 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:06.920508 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:06.920523 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:06.920777 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:06.920824 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 16:58:06.920853 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
I20260619 16:58:06.921732 20542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:06.922626 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:06.926911 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:06.923956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59743 (local address 127.30.148.190:34901)
0619 16:58:06.923989 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:06.923991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:06.924078 (+ 87us) server_negotiation.cc:407] Connection header received
0619 16:58:06.924106 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:06.924107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:06.924116 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:06.924191 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:06.924280 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:06.924974 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:06.925704 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:06.926021 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:06.926096 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:06.926680 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:06.926682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:06.926682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:06.926686 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:06.926706 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:06.926710 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:06.926780 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:06.926835 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 16:58:06.926845 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:58:06.930015 20542 catalog_manager.cc:1347] Generated new cluster ID: 7e9ebc0589dd400893544a6b6382f6d3
I20260619 16:58:06.930083 20542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:06.946801 20542 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:06.947472 20542 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:06.950759 20542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66260414de7f41039c67ca7020e46b39: Generated new TSK 0
I20260619 16:58:06.950927 20542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:06.956702 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 20460
W20260619 16:58:06.960716 20445 connection.cc:535] client connection to 127.30.148.190:34901 recv error: Network error: failed to read from TLS socket (remote: 127.30.148.190:34901): Connection reset by peer (error 104)
2026-06-19T16:58:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1303 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:07 dist-test-slave-xb9n krb5kdc[20559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:07 dist-test-slave-xb9n krb5kdc[20559](info): set up 2 sockets
Jun 19 16:58:07 dist-test-slave-xb9n krb5kdc[20559](info): commencing operation
krb5kdc: starting...
W20260619 16:58:07.993430 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:58:08 dist-test-slave-xb9n krb5kdc[20559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:58:08.104225 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36153
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:45659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:36153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:08.181908 20572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:08.182098 20572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:08.182121 20572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:08.183120 20572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:08.183153 20572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:08.183172 20572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:08.183185 20572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:08.183197 20572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:08.184502 20572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:36153
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:08.184671 20572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:08.184821 20572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:08.189291 20578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:08.189304 20580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:08.189371 20572 server_base.cc:1034] running on GCE node
W20260619 16:58:08.189304 20577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:08.189812 20572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:08.190030 20572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:08.191193 20572 hybrid_clock.cc:648] HybridClock initialized: now 1781888288191167 us; error 39 us; skew 500 ppm
Jun 19 16:58:08 dist-test-slave-xb9n krb5kdc[20559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888288, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:08.193548 20572 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:58:08.193940 20572 webserver.cc:460] Webserver started at http://127.30.148.190:40069/ using document root <none> and password file <none>
I20260619 16:58:08.194162 20572 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:08.194227 20572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:08.194371 20572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:08.195340 20572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "2dc31947b92640c184308df5d2ad22d9"
format_stamp: "Formatted at 2026-06-19 16:58:08 on dist-test-slave-xb9n"
server_key: "6581032904ea995695f48e4c1ab01877"
server_key_iv: "6419fd976057223f6283016b648fe91f"
server_key_version: "encryptionkey@0"
I20260619 16:58:08.195689 20572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "2dc31947b92640c184308df5d2ad22d9"
format_stamp: "Formatted at 2026-06-19 16:58:08 on dist-test-slave-xb9n"
server_key: "6581032904ea995695f48e4c1ab01877"
server_key_iv: "6419fd976057223f6283016b648fe91f"
server_key_version: "encryptionkey@0"
I20260619 16:58:08.196992 20572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:08.197798 20587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:08.197965 20572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:08.198050 20572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "2dc31947b92640c184308df5d2ad22d9"
format_stamp: "Formatted at 2026-06-19 16:58:08 on dist-test-slave-xb9n"
server_key: "6581032904ea995695f48e4c1ab01877"
server_key_iv: "6419fd976057223f6283016b648fe91f"
server_key_version: "encryptionkey@0"
I20260619 16:58:08.198127 20572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:08.201256 20572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:08.202980 20572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:08.203130 20572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:08.206949 20572 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:36153
I20260619 16:58:08.206980 20639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:36153 every 8 connection(s)
I20260619 16:58:08.207301 20572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:08.207953 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 20572
I20260619 16:58:08.208081 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:58:08.208271 31314 external_mini_cluster.cc:1428] Setting key 4fab29032ec0b37cbfdea466309a325d
I20260619 16:58:08.208348 20640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:08.210813 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:08.208754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50695 (local address 127.30.148.190:36153)
0619 16:58:08.208936 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:08.208945 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:08.208977 (+ 32us) server_negotiation.cc:407] Connection header received
0619 16:58:08.209538 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:08.209547 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:08.209580 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:08.209737 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:08.210313 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:08.210317 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:08.210318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:08.210322 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:08.210363 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:08.210376 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:08.210454 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:08.210487 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 16:58:08.210619 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260619 16:58:08.211647 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:08.211844 20640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9: Bootstrap starting.
I20260619 16:58:08.212401 20640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:08.212719 20640 log.cc:824] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:08.213584 20640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9: No bootstrap required, opened a new log
I20260619 16:58:08.214237 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:08.212922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37769 (local address 127.30.148.190:36153)
0619 16:58:08.212953 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:08.212955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:08.213331 (+ 376us) server_negotiation.cc:407] Connection header received
0619 16:58:08.213391 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:08.213392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:08.213396 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:08.213446 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:08.214046 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:08.214047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:08.214047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:08.214052 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:08.214065 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:08.214069 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:08.214135 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:08.214171 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 16:58:08.214183 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 16:58:08.215440 20640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc31947b92640c184308df5d2ad22d9" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36153 } }
I20260619 16:58:08.215559 20640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:08.215580 20640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc31947b92640c184308df5d2ad22d9, State: Initialized, Role: FOLLOWER
I20260619 16:58:08.215651 20640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc31947b92640c184308df5d2ad22d9" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36153 } }
I20260619 16:58:08.215718 20640 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:08.215739 20640 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:08.215764 20640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:08.216208 20640 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc31947b92640c184308df5d2ad22d9" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36153 } }
I20260619 16:58:08.216287 20640 leader_election.cc:302] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dc31947b92640c184308df5d2ad22d9; no voters:
I20260619 16:58:08.216394 20640 leader_election.cc:288] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:08.216470 20650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:08.216603 20650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [term 1 LEADER]: Becoming Leader. State: Replica: 2dc31947b92640c184308df5d2ad22d9, State: Running, Role: LEADER
I20260619 16:58:08.216696 20650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc31947b92640c184308df5d2ad22d9" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36153 } }
I20260619 16:58:08.216837 20640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:08.217067 20651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dc31947b92640c184308df5d2ad22d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc31947b92640c184308df5d2ad22d9" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36153 } } }
I20260619 16:58:08.217089 20652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dc31947b92640c184308df5d2ad22d9. Latest consensus state: current_term: 1 leader_uuid: "2dc31947b92640c184308df5d2ad22d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dc31947b92640c184308df5d2ad22d9" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36153 } } }
I20260619 16:58:08.217305 20652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:08.217221 20651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:08.217780 20657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:08.218256 20657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:08.219633 20657 catalog_manager.cc:1347] Generated new cluster ID: 1ea8f0f4c1b74de097dc110eae277b6b
I20260619 16:58:08.219688 20657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:08.225915 20657 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:08.226052 20657 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:08.228861 20657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dc31947b92640c184308df5d2ad22d9: Generated new TSK 0
I20260619 16:58:08.229007 20657 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:08.235606 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 20572
2026-06-19T16:58:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1278 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:08 dist-test-slave-xb9n krb5kdc[20671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:08 dist-test-slave-xb9n krb5kdc[20671](info): set up 2 sockets
Jun 19 16:58:08 dist-test-slave-xb9n krb5kdc[20671](info): commencing operation
krb5kdc: starting...
W20260619 16:58:09.257565 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:58:09 dist-test-slave-xb9n krb5kdc[20671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:58:09.369271 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:45467
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:45467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:09.446481 20684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:09.446712 20684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:09.446743 20684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:09.447844 20684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:09.447916 20684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:09.447939 20684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:09.447961 20684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:09.447988 20684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:09.449455 20684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:45467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:45467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:09.449697 20684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:09.449901 20684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:09.454591 20692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:09.454634 20690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:09.454710 20684 server_base.cc:1034] running on GCE node
W20260619 16:58:09.454628 20689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:09.455139 20684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:09.455436 20684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:09.456598 20684 hybrid_clock.cc:648] HybridClock initialized: now 1781888289456588 us; error 47 us; skew 500 ppm
Jun 19 16:58:09 dist-test-slave-xb9n krb5kdc[20671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888289, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:09.462872 20684 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:58:09.463231 20684 webserver.cc:460] Webserver started at http://127.30.148.190:36785/ using document root <none> and password file <none>
I20260619 16:58:09.463399 20684 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:09.463438 20684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:09.463519 20684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:09.464468 20684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "706167e10531477199917e9bff2c3cf7"
format_stamp: "Formatted at 2026-06-19 16:58:09 on dist-test-slave-xb9n"
server_key: "8d6c76c25d4b07733dc1a8923aafcc06"
server_key_iv: "e175f9cb016fd40e57513c74bb019514"
server_key_version: "encryptionkey@0"
I20260619 16:58:09.464766 20684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "706167e10531477199917e9bff2c3cf7"
format_stamp: "Formatted at 2026-06-19 16:58:09 on dist-test-slave-xb9n"
server_key: "8d6c76c25d4b07733dc1a8923aafcc06"
server_key_iv: "e175f9cb016fd40e57513c74bb019514"
server_key_version: "encryptionkey@0"
I20260619 16:58:09.466040 20684 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:09.466763 20699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:09.466964 20684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:09.467048 20684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "706167e10531477199917e9bff2c3cf7"
format_stamp: "Formatted at 2026-06-19 16:58:09 on dist-test-slave-xb9n"
server_key: "8d6c76c25d4b07733dc1a8923aafcc06"
server_key_iv: "e175f9cb016fd40e57513c74bb019514"
server_key_version: "encryptionkey@0"
I20260619 16:58:09.467103 20684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:09.483110 20684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:09.485304 20684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:09.485481 20684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:09.489013 20684 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:45467
I20260619 16:58:09.489068 20751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:45467 every 8 connection(s)
I20260619 16:58:09.489395 20684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:09.490218 20752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:09.492317 20752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7: Bootstrap starting.
I20260619 16:58:09.492859 20752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:09.493177 20752 log.cc:824] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:09.493247 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 20684
I20260619 16:58:09.493347 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:58:09.493547 31314 external_mini_cluster.cc:1428] Setting key a7465ce877612d5917eb82b81085e62c
I20260619 16:58:09.494021 20752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7: No bootstrap required, opened a new log
I20260619 16:58:09.495666 20752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706167e10531477199917e9bff2c3cf7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 45467 } }
I20260619 16:58:09.495826 20752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:09.495864 20752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 706167e10531477199917e9bff2c3cf7, State: Initialized, Role: FOLLOWER
I20260619 16:58:09.496054 20752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706167e10531477199917e9bff2c3cf7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 45467 } }
I20260619 16:58:09.496150 20752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:09.496184 20752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:09.496235 20752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:09.497184 20752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706167e10531477199917e9bff2c3cf7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 45467 } }
I20260619 16:58:09.497306 20752 leader_election.cc:302] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 706167e10531477199917e9bff2c3cf7; no voters:
I20260619 16:58:09.497502 20752 leader_election.cc:288] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:09.497548 20757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:09.497735 20757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [term 1 LEADER]: Becoming Leader. State: Replica: 706167e10531477199917e9bff2c3cf7, State: Running, Role: LEADER
I20260619 16:58:09.497785 20752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:09.497913 20757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706167e10531477199917e9bff2c3cf7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 45467 } }
I20260619 16:58:09.498323 20759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 706167e10531477199917e9bff2c3cf7. Latest consensus state: current_term: 1 leader_uuid: "706167e10531477199917e9bff2c3cf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706167e10531477199917e9bff2c3cf7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 45467 } } }
I20260619 16:58:09.498340 20758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "706167e10531477199917e9bff2c3cf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706167e10531477199917e9bff2c3cf7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 45467 } } }
I20260619 16:58:09.498473 20758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:09.498649 20759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:09.498684 20766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:09.499282 20766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:09.501030 20766 catalog_manager.cc:1347] Generated new cluster ID: 6e4a6f7a18fe4efeb634def680931d37
I20260619 16:58:09.501101 20766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:58:09 dist-test-slave-xb9n krb5kdc[20671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:58:09.505625 20756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:09.494034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47783 (local address 127.30.148.190:45467)
0619 16:58:09.494221 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:09.494229 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:09.494252 (+ 23us) server_negotiation.cc:407] Connection header received
0619 16:58:09.495647 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:09.495654 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:09.495701 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:09.495811 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:09.496634 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:09.497703 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:09.498801 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:09.499445 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:09.499606 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:09.502282 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:09.502286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:09.502286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:09.502297 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:09.503948 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:09.504328 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:09.504330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:09.504331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:09.504441 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:09.504660 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:09.504660 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:09.504661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:09.504899 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:09.505076 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:09.505140 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 16:58:09.505278 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
I20260619 16:58:09.506584 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:09.515509 20756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:09.509686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35065 (local address 127.30.148.190:45467)
0619 16:58:09.509732 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:09.509734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:09.509790 (+ 56us) server_negotiation.cc:407] Connection header received
0619 16:58:09.509858 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:09.509859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:09.509870 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:09.509924 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:09.510430 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:09.510891 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:09.511593 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:09.511826 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:09.511902 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:09.513036 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:09.513037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:09.513037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:09.513044 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:09.514425 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:09.514800 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:09.514801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:09.514801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:09.514853 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:09.515043 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:09.515044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:09.515044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:09.515146 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:09.515257 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:09.515354 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 16:58:09.515421 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 16:58:09.523895 20766 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:09.524565 20766 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:09.536556 20766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 706167e10531477199917e9bff2c3cf7: Generated new TSK 0
I20260619 16:58:09.536746 20766 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:09.546149 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 20684
2026-06-19T16:58:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:09 dist-test-slave-xb9n krb5kdc[20783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:09 dist-test-slave-xb9n krb5kdc[20783](info): set up 2 sockets
Jun 19 16:58:09 dist-test-slave-xb9n krb5kdc[20783](info): commencing operation
krb5kdc: starting...
W20260619 16:58:10.577440 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:58:10 dist-test-slave-xb9n krb5kdc[20783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:58:10.692520 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33115
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:45027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:33115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:10.773274 20796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:10.773556 20796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:10.773592 20796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:10.774714 20796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:10.774777 20796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:10.774803 20796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:10.774834 20796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:10.774863 20796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:10.776266 20796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:33115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:33115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:10.776535 20796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:10.776746 20796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:10.781213 20801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:10.781405 20796 server_base.cc:1034] running on GCE node
W20260619 16:58:10.781232 20804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:10.781211 20802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:10.781724 20796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:10.781997 20796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:10.783178 20796 hybrid_clock.cc:648] HybridClock initialized: now 1781888290783144 us; error 39 us; skew 500 ppm
Jun 19 16:58:10 dist-test-slave-xb9n krb5kdc[20783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888290, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:10.785504 20796 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:58:10.785847 20796 webserver.cc:460] Webserver started at http://127.30.148.190:44995/ using document root <none> and password file <none>
I20260619 16:58:10.786063 20796 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:10.786103 20796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:10.786208 20796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:10.787106 20796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "c729f6d3718d4e4784c344dc77c0b727"
format_stamp: "Formatted at 2026-06-19 16:58:10 on dist-test-slave-xb9n"
server_key: "e71d85bb42b49598630f405b8156a90c"
server_key_iv: "f09d742663c89dad311597169ee08a6d"
server_key_version: "encryptionkey@0"
I20260619 16:58:10.787397 20796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "c729f6d3718d4e4784c344dc77c0b727"
format_stamp: "Formatted at 2026-06-19 16:58:10 on dist-test-slave-xb9n"
server_key: "e71d85bb42b49598630f405b8156a90c"
server_key_iv: "f09d742663c89dad311597169ee08a6d"
server_key_version: "encryptionkey@0"
I20260619 16:58:10.788581 20796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:58:10.789180 20811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:10.789328 20796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:10.789422 20796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "c729f6d3718d4e4784c344dc77c0b727"
format_stamp: "Formatted at 2026-06-19 16:58:10 on dist-test-slave-xb9n"
server_key: "e71d85bb42b49598630f405b8156a90c"
server_key_iv: "f09d742663c89dad311597169ee08a6d"
server_key_version: "encryptionkey@0"
I20260619 16:58:10.789484 20796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:10.805831 20796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:10.807534 20796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:10.807643 20796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:10.811266 20796 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:33115
I20260619 16:58:10.811313 20863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:33115 every 8 connection(s)
I20260619 16:58:10.811630 20796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:10.812433 20864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:10.814668 20864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727: Bootstrap starting.
I20260619 16:58:10.815228 20864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:10.815491 20864 log.cc:824] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:10.816182 20864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727: No bootstrap required, opened a new log
I20260619 16:58:10.816425 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 20796
I20260619 16:58:10.816545 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:58:10.816745 31314 external_mini_cluster.cc:1428] Setting key cd37af91689ebfb249256a71ab7c8326
I20260619 16:58:10.818403 20864 raft_consensus.cc:348] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c729f6d3718d4e4784c344dc77c0b727" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33115 } }
I20260619 16:58:10.818547 20864 raft_consensus.cc:374] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:10.818598 20864 raft_consensus.cc:729] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c729f6d3718d4e4784c344dc77c0b727, State: Initialized, Role: FOLLOWER
I20260619 16:58:10.818744 20864 consensus_queue.cc:260] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c729f6d3718d4e4784c344dc77c0b727" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33115 } }
I20260619 16:58:10.818828 20864 raft_consensus.cc:388] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:10.818871 20864 raft_consensus.cc:482] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:10.818917 20864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:10.819403 20864 raft_consensus.cc:504] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c729f6d3718d4e4784c344dc77c0b727" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33115 } }
I20260619 16:58:10.819516 20864 leader_election.cc:302] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c729f6d3718d4e4784c344dc77c0b727; no voters:
I20260619 16:58:10.819689 20864 leader_election.cc:288] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:10.819711 20869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:10.819847 20869 raft_consensus.cc:686] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [term 1 LEADER]: Becoming Leader. State: Replica: c729f6d3718d4e4784c344dc77c0b727, State: Running, Role: LEADER
I20260619 16:58:10.819950 20864 sys_catalog.cc:564] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:10.819973 20869 consensus_queue.cc:237] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c729f6d3718d4e4784c344dc77c0b727" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33115 } }
I20260619 16:58:10.820310 20869 sys_catalog.cc:455] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c729f6d3718d4e4784c344dc77c0b727. Latest consensus state: current_term: 1 leader_uuid: "c729f6d3718d4e4784c344dc77c0b727" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c729f6d3718d4e4784c344dc77c0b727" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33115 } } }
I20260619 16:58:10.820322 20871 sys_catalog.cc:455] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c729f6d3718d4e4784c344dc77c0b727" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c729f6d3718d4e4784c344dc77c0b727" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 33115 } } }
I20260619 16:58:10.820444 20869 sys_catalog.cc:458] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:10.820765 20878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:10.821252 20878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:10.820509 20871 sys_catalog.cc:458] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:10.823127 20878 catalog_manager.cc:1347] Generated new cluster ID: 7017d2a34f8641edae171f416a16bff8
I20260619 16:58:10.823181 20878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:10.823408 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:10.817251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40223 (local address 127.30.148.190:33115)
0619 16:58:10.817464 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:10.817471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:10.817493 (+ 22us) server_negotiation.cc:407] Connection header received
0619 16:58:10.817946 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:10.817953 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:10.817993 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:10.818101 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:10.818347 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:10.819543 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:10.820593 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:10.821231 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:10.821348 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:10.822555 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:10.822558 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:10.822559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:10.822762 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:10.822801 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:10.822816 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:10.822877 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:10.823137 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 16:58:10.823202 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
I20260619 16:58:10.826264 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:10.838512 20878 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:10.839147 20878 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:10.840655 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:10.829628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51577 (local address 127.30.148.190:33115)
0619 16:58:10.829656 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:10.829658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:10.837514 (+ 7856us) server_negotiation.cc:407] Connection header received
0619 16:58:10.837610 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:10.837611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:10.837620 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:10.837648 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:10.837850 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:10.838410 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:10.839380 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:10.839676 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:10.839708 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:10.840419 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:10.840420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:10.840421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:10.840424 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:10.840437 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:10.840441 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:10.840466 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:10.840559 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 16:58:10.840588 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 16:58:10.844949 20878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c729f6d3718d4e4784c344dc77c0b727: Generated new TSK 0
I20260619 16:58:10.845120 20878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:10.850230 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 20796
2026-06-19T16:58:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:10 dist-test-slave-xb9n krb5kdc[20895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:10 dist-test-slave-xb9n krb5kdc[20895](info): set up 2 sockets
Jun 19 16:58:10 dist-test-slave-xb9n krb5kdc[20895](info): commencing operation
krb5kdc: starting...
W20260619 16:58:11.877547 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:58:11 dist-test-slave-xb9n krb5kdc[20895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:58:11.988737 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:38737
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:38737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:12.067088 20908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:12.067323 20908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:12.067359 20908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:12.068500 20908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:12.068568 20908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:12.068593 20908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:12.068614 20908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:12.068642 20908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:12.070097 20908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:38737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:38737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:12.070374 20908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:12.070591 20908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:58:12.075243 20908 server_base.cc:1034] running on GCE node
W20260619 16:58:12.075359 20914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:12.075253 20916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:12.075254 20913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:12.075693 20908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:12.075977 20908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:12.077158 20908 hybrid_clock.cc:648] HybridClock initialized: now 1781888292077152 us; error 35 us; skew 500 ppm
Jun 19 16:58:12 dist-test-slave-xb9n krb5kdc[20895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888292, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:12.079571 20908 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:58:12.079978 20908 webserver.cc:460] Webserver started at http://127.30.148.190:39707/ using document root <none> and password file <none>
I20260619 16:58:12.080219 20908 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:12.080286 20908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:12.080436 20908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:12.081526 20908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "74015f6d8c7145d281279f9efa8d4af7"
format_stamp: "Formatted at 2026-06-19 16:58:12 on dist-test-slave-xb9n"
server_key: "2f3d866374453bd64a3243cc14a98225"
server_key_iv: "e56e4de79784beb6c5d3fe74b8a3099e"
server_key_version: "encryptionkey@0"
I20260619 16:58:12.081882 20908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "74015f6d8c7145d281279f9efa8d4af7"
format_stamp: "Formatted at 2026-06-19 16:58:12 on dist-test-slave-xb9n"
server_key: "2f3d866374453bd64a3243cc14a98225"
server_key_iv: "e56e4de79784beb6c5d3fe74b8a3099e"
server_key_version: "encryptionkey@0"
I20260619 16:58:12.083334 20908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:58:12.084023 20923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:12.084178 20908 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:58:12.084290 20908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "74015f6d8c7145d281279f9efa8d4af7"
format_stamp: "Formatted at 2026-06-19 16:58:12 on dist-test-slave-xb9n"
server_key: "2f3d866374453bd64a3243cc14a98225"
server_key_iv: "e56e4de79784beb6c5d3fe74b8a3099e"
server_key_version: "encryptionkey@0"
I20260619 16:58:12.084360 20908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:12.087626 20908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:12.089469 20908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:12.089612 20908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:12.093490 20908 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:38737
I20260619 16:58:12.093528 20975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:38737 every 8 connection(s)
I20260619 16:58:12.093822 20908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:12.094748 20976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:12.096791 20976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7: Bootstrap starting.
I20260619 16:58:12.097273 20976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:12.097574 20976 log.cc:824] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:12.098208 20976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7: No bootstrap required, opened a new log
I20260619 16:58:12.099699 20976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74015f6d8c7145d281279f9efa8d4af7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38737 } }
I20260619 16:58:12.099823 20976 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:12.099845 20976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74015f6d8c7145d281279f9efa8d4af7, State: Initialized, Role: FOLLOWER
I20260619 16:58:12.100056 20976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74015f6d8c7145d281279f9efa8d4af7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38737 } }
I20260619 16:58:12.100159 20976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:12.100186 20976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:12.100215 20976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:12.100962 20976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74015f6d8c7145d281279f9efa8d4af7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38737 } }
I20260619 16:58:12.101042 20976 leader_election.cc:302] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74015f6d8c7145d281279f9efa8d4af7; no voters:
I20260619 16:58:12.101271 20976 leader_election.cc:288] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:12.101289 20979 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:12.101481 20979 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [term 1 LEADER]: Becoming Leader. State: Replica: 74015f6d8c7145d281279f9efa8d4af7, State: Running, Role: LEADER
I20260619 16:58:12.101612 20976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:12.101616 20979 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74015f6d8c7145d281279f9efa8d4af7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38737 } }
I20260619 16:58:12.102005 20981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74015f6d8c7145d281279f9efa8d4af7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74015f6d8c7145d281279f9efa8d4af7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38737 } } }
I20260619 16:58:12.102044 20980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74015f6d8c7145d281279f9efa8d4af7. Latest consensus state: current_term: 1 leader_uuid: "74015f6d8c7145d281279f9efa8d4af7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74015f6d8c7145d281279f9efa8d4af7" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 38737 } } }
I20260619 16:58:12.102197 20980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:12.102408 20981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:12.102558 20988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:12.102656 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 20908
I20260619 16:58:12.102792 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:58:12.103020 31314 external_mini_cluster.cc:1428] Setting key 0517ac495e6f11fc601869e63e83a80f
I20260619 16:58:12.103075 20988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:12.105535 20998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:12.103607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47051 (local address 127.30.148.190:38737)
0619 16:58:12.104216 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:12.104222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:12.104245 (+ 23us) server_negotiation.cc:407] Connection header received
0619 16:58:12.104329 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:12.104334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:12.104361 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:12.104492 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:12.105117 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:12.105120 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:12.105120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:12.105124 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:12.105170 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:12.105182 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:12.105258 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:12.105328 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 16:58:12.105393 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":72,"threads_started":1}
I20260619 16:58:12.106436 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:12.108729 20998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:12.107692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53873 (local address 127.30.148.190:38737)
0619 16:58:12.107758 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:12.107760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:12.107850 (+ 90us) server_negotiation.cc:407] Connection header received
0619 16:58:12.107933 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:12.107934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:12.107939 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:12.107993 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:12.108531 (+ 538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:12.108532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:12.108532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:12.108536 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:12.108548 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:12.108551 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:12.108598 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:12.108649 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 16:58:12.108663 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 16:58:12.109681 20988 catalog_manager.cc:1347] Generated new cluster ID: 1a8bf64e09124153bd4c87366b7e79c5
I20260619 16:58:12.109740 20988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:12.115368 20988 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:12.115514 20988 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:12.118228 20988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74015f6d8c7145d281279f9efa8d4af7: Generated new TSK 0
I20260619 16:58:12.118376 20988 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:12.123104 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 20908
2026-06-19T16:58:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1272 ms)
[----------] 18 tests from AuthTokenIssuingTest (23335 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:12 dist-test-slave-xb9n krb5kdc[21007](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:12 dist-test-slave-xb9n krb5kdc[21007](info): set up 2 sockets
Jun 19 16:58:12 dist-test-slave-xb9n krb5kdc[21007](info): commencing operation
krb5kdc: starting...
W20260619 16:58:13.145318 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:58:13.256587 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36781
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:36781,127.30.148.189:44219,127.30.148.188:36505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:13.334386 21020 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 16:58:13.334620 21020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:13.334650 21020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:13.334692 21020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:13.335793 21020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:13.335858 21020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:13.335882 21020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:13.335911 21020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:13.335937 21020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:13.337381 21020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:36781,127.30.148.189:44219,127.30.148.188:36505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:36781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:13.337642 21020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:13.337862 21020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:13.342374 21028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:13.342454 21020 server_base.cc:1034] running on GCE node
W20260619 16:58:13.342381 21025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:13.342379 21026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:13.342823 21020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:13.343101 21020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:13.344261 21020 hybrid_clock.cc:648] HybridClock initialized: now 1781888293344258 us; error 36 us; skew 500 ppm
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:13.346793 21020 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:58:13.347231 21020 webserver.cc:460] Webserver started at http://127.30.148.190:38323/ using document root <none> and password file <none>
I20260619 16:58:13.347404 21020 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:13.347441 21020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:13.347525 21020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:13.348479 21020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "868543a1b73e43139eb12cb8b21c4c41"
format_stamp: "Formatted at 2026-06-19 16:58:13 on dist-test-slave-xb9n"
server_key: "cb880fd477e7cdb2717dbbba11e1b349"
server_key_iv: "2a4cf3f83b2f1c6f4cc2d956544f328a"
server_key_version: "encryptionkey@0"
I20260619 16:58:13.348763 21020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "868543a1b73e43139eb12cb8b21c4c41"
format_stamp: "Formatted at 2026-06-19 16:58:13 on dist-test-slave-xb9n"
server_key: "cb880fd477e7cdb2717dbbba11e1b349"
server_key_iv: "2a4cf3f83b2f1c6f4cc2d956544f328a"
server_key_version: "encryptionkey@0"
I20260619 16:58:13.350026 21020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:13.350688 21035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:13.350886 21020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:58:13.350968 21020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "868543a1b73e43139eb12cb8b21c4c41"
format_stamp: "Formatted at 2026-06-19 16:58:13 on dist-test-slave-xb9n"
server_key: "cb880fd477e7cdb2717dbbba11e1b349"
server_key_iv: "2a4cf3f83b2f1c6f4cc2d956544f328a"
server_key_version: "encryptionkey@0"
I20260619 16:58:13.351033 21020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:13.370093 21020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:13.371850 21020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:13.371958 21020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:13.375612 21020 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:36781
I20260619 16:58:13.375689 21087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:36781 every 8 connection(s)
I20260619 16:58:13.376005 21020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:13.376853 21088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:13.378392 21088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:58:13.380553 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 21020
I20260619 16:58:13.380710 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:58:13.380904 31314 external_mini_cluster.cc:1428] Setting key e1a225fe5dcde7985b5791903bcb9963
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.148.189@KRBTEST.COM; defaulting to no policy
I20260619 16:58:13.388899 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.379034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36781 (local address 127.0.0.1:33626)
0619 16:58:13.379293 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.379332 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:13.379790 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:13.380074 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:13.380076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:13.380348 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:13.380489 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.380493 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.381718 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.381719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.382766 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.382769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.383345 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.383346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.383404 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.383904 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:13.383912 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:13.385269 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:13.387435 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.387441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.387443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.387625 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.387894 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.387894 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.387895 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.387970 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.388432 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:13.388437 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:13.388489 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 16:58:13.388562 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 16:58:13.388676 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":114,"threads_started":1}
I20260619 16:58:13.388904 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.379055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.30.148.190:36781)
0619 16:58:13.379228 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.379237 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.379412 (+ 175us) server_negotiation.cc:407] Connection header received
0619 16:58:13.379939 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.379942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.379966 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.380066 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.380594 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.381623 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.382860 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.383255 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.383373 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.385397 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.385405 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.385406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.385413 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.387349 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.387712 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.387715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.387715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.387817 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.388038 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.388039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.388040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.388355 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.388445 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.388619 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.388752 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260619 16:58:13.389619 21088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:58:13.389931 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.389840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:44219 (local address 127.0.0.1:56498)
0619 16:58:13.389877 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.389915 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:44219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260619 16:58:13.390067 21037 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.148.189:44219: connect: Connection refused (error 111)
I20260619 16:58:13.390431 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.390377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:44219 (local address 127.0.0.1:56502)
0619 16:58:13.390398 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.390418 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:44219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260619 16:58:13.390540 21088 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.189:44219: Network error: Client connection negotiation failed: client connection to 127.30.148.189:44219: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.189.keytab.
Entry for principal kudu/127.30.148.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.189.keytab.
I20260619 16:58:13.405561 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.189:44219
--webserver_interface=127.30.148.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.189.keytab
--principal=kudu/127.30.148.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:36781,127.30.148.189:44219,127.30.148.188:36505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:58:13.439745 21088 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } attempt: 1
I20260619 16:58:13.440289 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.440130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:44219 (local address 127.0.0.1:56518)
0619 16:58:13.440217 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.440256 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:44219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 16:58:13.440661 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.440565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:44219 (local address 127.0.0.1:56524)
0619 16:58:13.440611 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.440641 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:44219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 16:58:13.440759 21088 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.189:44219: Network error: Client connection negotiation failed: client connection to 127.30.148.189:44219: connect: Connection refused (error 111)
W20260619 16:58:13.487262 21095 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 16:58:13.487481 21095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:13.487510 21095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:13.487566 21095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:13.488693 21095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:13.488756 21095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:13.488782 21095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:13.488813 21095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:13.488842 21095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:13.490335 21095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:36781,127.30.148.189:44219,127.30.148.188:36505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.189.keytab
--principal=kudu/127.30.148.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.189:44219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.148.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:13.490592 21095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:13.490820 21095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:13.495442 21100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:13.495492 21095 server_base.cc:1034] running on GCE node
W20260619 16:58:13.495455 21103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:13.495517 21101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:13.495879 21095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:13.496125 21095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:13.497258 21095 hybrid_clock.cc:648] HybridClock initialized: now 1781888293497250 us; error 27 us; skew 500 ppm
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:13.499574 21095 init.cc:377] Logged in from keytab as kudu/127.30.148.189@KRBTEST.COM (short username kudu)
I20260619 16:58:13.500018 21095 webserver.cc:460] Webserver started at http://127.30.148.189:36621/ using document root <none> and password file <none>
I20260619 16:58:13.500242 21095 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:13.500313 21095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:13.500458 21095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:13.501505 21095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/data/instance:
uuid: "a40cd16f203d49b6bbb9cf671dbea6d9"
format_stamp: "Formatted at 2026-06-19 16:58:13 on dist-test-slave-xb9n"
server_key: "473cd072fb38150fb2b6709608791d00"
server_key_iv: "c0283753d1cef99a2c43d58c8a0b30bf"
server_key_version: "encryptionkey@0"
I20260619 16:58:13.501852 21095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/wal/instance:
uuid: "a40cd16f203d49b6bbb9cf671dbea6d9"
format_stamp: "Formatted at 2026-06-19 16:58:13 on dist-test-slave-xb9n"
server_key: "473cd072fb38150fb2b6709608791d00"
server_key_iv: "c0283753d1cef99a2c43d58c8a0b30bf"
server_key_version: "encryptionkey@0"
I20260619 16:58:13.503108 21095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:13.503826 21110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:13.503988 21095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:13.504108 21095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/wal
uuid: "a40cd16f203d49b6bbb9cf671dbea6d9"
format_stamp: "Formatted at 2026-06-19 16:58:13 on dist-test-slave-xb9n"
server_key: "473cd072fb38150fb2b6709608791d00"
server_key_iv: "c0283753d1cef99a2c43d58c8a0b30bf"
server_key_version: "encryptionkey@0"
I20260619 16:58:13.504195 21095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:13.508900 21088 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } attempt: 2
I20260619 16:58:13.509279 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.509152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:44219 (local address 127.0.0.1:56526)
0619 16:58:13.509214 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.509252 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:44219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 16:58:13.509654 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.509566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:44219 (local address 127.0.0.1:56528)
0619 16:58:13.509612 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.509637 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:44219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 16:58:13.509760 21088 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.189:44219: Network error: Client connection negotiation failed: client connection to 127.30.148.189:44219: connect: Connection refused (error 111)
I20260619 16:58:13.516692 21095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:13.518487 21095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:13.518640 21095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:13.522557 21095 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.189:44219
I20260619 16:58:13.522589 21162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.189:44219 every 8 connection(s)
I20260619 16:58:13.522910 21095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
I20260619 16:58:13.523672 21163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:13.525091 21163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:58:13.529676 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 21095
I20260619 16:58:13.529793 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-1/wal/instance
I20260619 16:58:13.530001 31314 external_mini_cluster.cc:1428] Setting key 6d16fa58d1123f25989c5abc2253372a
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.189@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:58:13.533768 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.525725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33636 (local address 127.30.148.190:36781)
0619 16:58:13.525771 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.525773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.525991 (+ 218us) server_negotiation.cc:407] Connection header received
0619 16:58:13.526487 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.526488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.526502 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.526557 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.527147 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.527623 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.528597 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.528799 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.528875 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.530966 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.530968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.530968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.530976 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.532385 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.532796 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.532797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.532798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.532858 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.533119 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.533121 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.533121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.533252 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.533326 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.533607 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.533687 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 16:58:13.533950 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.525702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36781 (local address 127.0.0.1:33636)
0619 16:58:13.525882 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.525913 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:13.526378 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:13.526627 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:13.526631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:13.526877 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:13.527053 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.527057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.527727 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.527728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.528497 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.528499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.528880 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.528882 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.529009 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.529520 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:13.529527 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:13.530874 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:13.532501 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.532507 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.532509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.532720 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.532923 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.532924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.532924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.533035 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.533316 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:13.533323 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:13.533482 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 16:58:13.533586 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 16:58:13.533732 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":85,"threads_started":1}
I20260619 16:58:13.534765 21163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.148.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.188@KRBTEST.COM" created.
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.189@KRBTEST.COM for kudu/127.30.148.189@KRBTEST.COM
I20260619 16:58:13.543586 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.535010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:44219 (local address 127.0.0.1:56542)
0619 16:58:13.535055 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.535065 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:13.535100 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:13.535357 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:13.535358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:13.535503 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:13.535571 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.535572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.536367 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.536368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.537223 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.537226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.537774 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.537775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.537830 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.538145 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:13.538149 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:13.540744 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:13.542575 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.542576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.542576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.542766 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.542996 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.542996 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.542997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.543046 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.543364 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:13.543365 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:13.543398 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 16:58:13.543449 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 16:58:13.543526 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 16:58:13.543747 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.535062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56542 (local address 127.30.148.189:44219)
0619 16:58:13.535203 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.535207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.535238 (+ 31us) server_negotiation.cc:407] Connection header received
0619 16:58:13.535255 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.535257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.535267 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.535328 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.535642 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.536264 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.537305 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.537694 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.537757 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.540837 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.540844 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.540845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.540858 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.542495 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.542861 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.542864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.542865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.542928 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.543108 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.543109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.543109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.543299 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.543381 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.543507 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.543597 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":91,"threads_started":1}
I20260619 16:58:13.544441 21163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:58:13.544792 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.544644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39396)
0619 16:58:13.544710 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.544760 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 16:58:13.544939 21112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
I20260619 16:58:13.545428 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.545300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39400)
0619 16:58:13.545351 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.545398 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 16:58:13.545542 21163 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:36505: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.188.keytab.
Entry for principal kudu/127.30.148.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.188.keytab.
I20260619 16:58:13.554183 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.188:36505
--webserver_interface=127.30.148.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.188.keytab
--principal=kudu/127.30.148.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:36781,127.30.148.189:44219,127.30.148.188:36505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:58:13.594775 21163 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } attempt: 1
I20260619 16:58:13.595288 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.595139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39412)
0619 16:58:13.595203 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.595255 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 16:58:13.595666 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.595563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39416)
0619 16:58:13.595622 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.595643 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 16:58:13.595789 21163 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:36505: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
I20260619 16:58:13.600903 21088 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } attempt: 3
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.189@KRBTEST.COM
I20260619 16:58:13.607846 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.601253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56554 (local address 127.30.148.189:44219)
0619 16:58:13.601294 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.601296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.601304 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:13.601417 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.601418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.601428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.601483 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.601801 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.602271 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.602962 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.603183 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.603282 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.605136 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.605137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.605138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.605145 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.606664 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.607083 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.607083 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.607084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.607135 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.607334 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.607334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.607335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.607433 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.607516 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.607679 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.607759 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:58:13.607872 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.601199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:44219 (local address 127.0.0.1:56554)
0619 16:58:13.601262 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.601272 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:13.601320 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:13.601489 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:13.601490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:13.601650 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:13.601713 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.601715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.602352 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.602353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.602878 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.602880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.603278 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.603279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.603369 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.603833 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:13.603839 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:13.605049 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:13.606751 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.606752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.606753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.606998 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.607197 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.607198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.607198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.607271 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.607513 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:13.607514 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:13.607566 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 16:58:13.607655 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 16:58:13.607776 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 16:58:13.608255 21088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:58:13.608580 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.608489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39418)
0619 16:58:13.608536 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.608563 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 16:58:13.608873 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.608788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39430)
0619 16:58:13.608830 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.608853 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 16:58:13.608963 21088 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:36505: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
W20260619 16:58:13.634516 21170 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 16:58:13.634718 21170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:13.634763 21170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:13.634802 21170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:13.635871 21170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:13.635931 21170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:13.635965 21170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:13.636021 21170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:13.636050 21170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:13.637509 21170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:36781,127.30.148.189:44219,127.30.148.188:36505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.188.keytab
--principal=kudu/127.30.148.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.188:36505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.148.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:13.637789 21170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:13.638005 21170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:58:13.640090 21088 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } attempt: 1
I20260619 16:58:13.640450 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.640338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39436)
0619 16:58:13.640395 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.640427 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 16:58:13.640802 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.640695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39448)
0619 16:58:13.640753 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.640778 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 16:58:13.640887 21088 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:36505: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
W20260619 16:58:13.642695 21175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:13.642688 21178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:13.642676 21176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:13.642705 21170 server_base.cc:1034] running on GCE node
I20260619 16:58:13.643198 21170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:13.643481 21170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:13.644615 21170 hybrid_clock.cc:648] HybridClock initialized: now 1781888293644605 us; error 26 us; skew 500 ppm
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:13.646955 21170 init.cc:377] Logged in from keytab as kudu/127.30.148.188@KRBTEST.COM (short username kudu)
I20260619 16:58:13.647380 21170 webserver.cc:460] Webserver started at http://127.30.148.188:34841/ using document root <none> and password file <none>
I20260619 16:58:13.647590 21170 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:13.647630 21170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:13.647805 21170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:13.648803 21170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/data/instance:
uuid: "99948ac7e2234ea4a50bb80211a028d9"
format_stamp: "Formatted at 2026-06-19 16:58:13 on dist-test-slave-xb9n"
server_key: "784aee0607840fa3755b54c6e553ce33"
server_key_iv: "5f06dd39d4bb54fc80ce24d617d63124"
server_key_version: "encryptionkey@0"
I20260619 16:58:13.649142 21170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/wal/instance:
uuid: "99948ac7e2234ea4a50bb80211a028d9"
format_stamp: "Formatted at 2026-06-19 16:58:13 on dist-test-slave-xb9n"
server_key: "784aee0607840fa3755b54c6e553ce33"
server_key_iv: "5f06dd39d4bb54fc80ce24d617d63124"
server_key_version: "encryptionkey@0"
I20260619 16:58:13.650444 21170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:13.651113 21185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:13.651299 21170 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:58:13.651362 21170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/wal
uuid: "99948ac7e2234ea4a50bb80211a028d9"
format_stamp: "Formatted at 2026-06-19 16:58:13 on dist-test-slave-xb9n"
server_key: "784aee0607840fa3755b54c6e553ce33"
server_key_iv: "5f06dd39d4bb54fc80ce24d617d63124"
server_key_version: "encryptionkey@0"
I20260619 16:58:13.651446 21170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:13.662310 21170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:13.663937 21163 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } attempt: 2
I20260619 16:58:13.664094 21170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:13.664281 21170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:13.664363 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.664220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39452)
0619 16:58:13.664289 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.664326 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 16:58:13.664760 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.664650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39454)
0619 16:58:13.664700 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.664730 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 16:58:13.664923 21163 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:36505: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36505: connect: Connection refused (error 111)
I20260619 16:58:13.668445 21170 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.188:36505
I20260619 16:58:13.668490 21237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.188:36505 every 8 connection(s)
I20260619 16:58:13.668844 21170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
I20260619 16:58:13.669739 21238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:13.671123 21238 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.188@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:58:13.678320 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 21170
I20260619 16:58:13.678443 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888036930978-31314-0/minicluster-data/master-2/wal/instance
I20260619 16:58:13.678627 31314 external_mini_cluster.cc:1428] Setting key 5260c42c2dae25895f717eeccf79e419
I20260619 16:58:13.679239 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.671720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.30.148.190:36781)
0619 16:58:13.671769 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.671771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.671976 (+ 205us) server_negotiation.cc:407] Connection header received
0619 16:58:13.672396 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.672397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.672406 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.672455 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.672974 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.673508 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.674383 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.674556 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.674605 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.676461 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.676463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.676463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.676469 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.677869 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.678208 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.678210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.678210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.678277 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.678551 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.678553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.678553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.678673 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.678757 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.679073 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.679151 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:58:13.679412 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.671704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:36781 (local address 127.0.0.1:33648)
0619 16:58:13.671860 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.671894 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:13.672319 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:13.672502 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:13.672505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:13.672722 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:13.672909 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.672913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.673579 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.673580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.674315 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.674317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.674625 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.674626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.674719 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.675154 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:13.675161 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:13.676381 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:13.677936 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.677942 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.677943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.678136 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.678344 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.678345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.678345 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.678489 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.678751 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:13.678758 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:13.678912 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 16:58:13.679046 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 16:58:13.679192 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260619 16:58:13.680119 21238 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.188@KRBTEST.COM
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.188@KRBTEST.COM for kudu/127.30.148.189@KRBTEST.COM
I20260619 16:58:13.687568 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.678989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.30.148.188:36505)
0619 16:58:13.679158 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.679162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.679182 (+ 20us) server_negotiation.cc:407] Connection header received
0619 16:58:13.679255 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.679258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.679271 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.679343 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.680163 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.680889 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.681898 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.682358 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.682430 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.684591 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.684599 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.684599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.684606 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.686393 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.686748 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.686751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.686751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.686808 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.687008 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.687009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.687009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.687148 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.687327 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.687370 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.687458 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260619 16:58:13.687706 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.680456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56570 (local address 127.30.148.189:44219)
0619 16:58:13.680500 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.680502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.680511 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:13.680576 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.680577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.680587 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.680642 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.680900 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.681629 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.682512 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.682783 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.682874 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.684984 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.684986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.684986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.684993 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.686453 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.686812 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.686813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.686813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.686871 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.687070 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.687071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.687071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.687185 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.687258 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.687417 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.687495 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:58:13.687965 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.680390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:44219 (local address 127.0.0.1:56570)
0619 16:58:13.680470 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.680480 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:13.680515 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:13.680646 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:13.680647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:13.680774 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:13.680822 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.680823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.681709 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.681710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.682437 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.682439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.682888 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.682889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.682957 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.683322 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:13.683327 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:13.684899 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:13.686549 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.686550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.686551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.686725 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.686939 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.686940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.686940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.687005 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.687262 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:13.687263 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:13.687304 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 16:58:13.687390 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 16:58:13.687513 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":36}
I20260619 16:58:13.688481 21238 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.188@KRBTEST.COM for kudu/127.30.148.188@KRBTEST.COM
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:58:13.695667 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.688726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39468)
0619 16:58:13.688768 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.688778 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:13.688811 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:13.688927 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:13.688928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:13.689063 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:13.689095 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.689096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.689971 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.689971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.690735 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.690737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.691222 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.691223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.691276 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.691721 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:13.691727 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:13.692924 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:13.694686 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.694687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.694688 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.694851 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.695056 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.695057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.695057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.695110 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.695354 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:13.695355 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:13.695392 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 16:58:13.695459 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 16:58:13.695553 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 16:58:13.695675 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.688766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39468 (local address 127.30.148.188:36505)
0619 16:58:13.688795 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.688797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.688805 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:13.688870 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.688871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.688878 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.688921 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.689162 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.689887 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.690837 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.691135 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.691193 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.693009 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.693010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.693010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.693017 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.694616 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.694936 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.694937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.694937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.694992 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.695185 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.695186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.695187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.695289 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.695361 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.695483 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.695600 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":14}
I20260619 16:58:13.696446 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.688414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33652 (local address 127.30.148.190:36781)
0619 16:58:13.688452 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.688454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.688467 (+ 13us) server_negotiation.cc:407] Connection header received
0619 16:58:13.688485 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.688486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.688493 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.688556 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.689164 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.689895 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.690839 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.691121 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.691193 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.693666 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.693668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.693668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.693675 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.695097 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.695453 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.695454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.695454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.695512 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.695709 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.695710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.695710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.695827 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.696018 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.696242 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.696322 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:58:13.697443 21238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9: Bootstrap starting.
I20260619 16:58:13.698455 21238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:13.698786 21238 log.cc:824] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:13.699565 21238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9: No bootstrap required, opened a new log
I20260619 16:58:13.700960 21238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } }
I20260619 16:58:13.701102 21238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99948ac7e2234ea4a50bb80211a028d9, State: Initialized, Role: FOLLOWER
I20260619 16:58:13.701190 21238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } }
I20260619 16:58:13.701550 21238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:13.701505 21244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } } }
I20260619 16:58:13.701609 21244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [sys.catalog]: This master's current role is: FOLLOWER
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.189@KRBTEST.COM
W20260619 16:58:13.703245 21255 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:58:13.703317 21255 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:58:13.704774 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.697114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56582 (local address 127.30.148.189:44219)
0619 16:58:13.697151 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.697153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.697175 (+ 22us) server_negotiation.cc:407] Connection header received
0619 16:58:13.697197 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.697198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.697207 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.697268 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.697825 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.698521 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.699408 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.699612 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.699687 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.702039 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.702041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.702041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.702048 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.703631 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.703984 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.703985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.703985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.704041 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.704251 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.704252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.704253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.704365 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.704545 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.704579 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.704646 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:58:13.705965 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:13.704088 21170 master_runner.cc:186] Error getting master registration for 127.30.148.190:36781: OK, instance_id { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" instance_seqno: 1781888293374564 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 16:58:13.706889 21170 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 16:58:13.707098 21062 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "868543a1b73e43139eb12cb8b21c4c41"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33652
I20260619 16:58:13.716017 21088 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } attempt: 2
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.188@KRBTEST.COM
I20260619 16:58:13.722630 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.716274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39470)
0619 16:58:13.716327 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.716339 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:13.716378 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:13.716537 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:13.716538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:13.716716 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:13.716760 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.716763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.717467 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.717468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.717965 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.717967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.718353 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.718354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.718411 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.718845 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:13.718850 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:13.719960 (+ 1110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:13.721610 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.721611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.721611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.721784 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.721984 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.721985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.721985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.722042 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.722321 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:13.722322 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:13.722365 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 16:58:13.722444 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 16:58:13.722538 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 16:58:13.722630 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.716356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39470 (local address 127.30.148.188:36505)
0619 16:58:13.716415 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.716417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.716426 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:13.716446 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.716447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.716456 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.716532 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.716833 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.717304 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.718063 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.718266 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.718353 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.720044 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.720045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.720046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.720052 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.721518 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.721857 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.721858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.721858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.721910 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.722113 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.722114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.722114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.722246 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.722317 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.722481 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.722549 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 16:58:13.723964 21088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41: Bootstrap starting.
I20260619 16:58:13.724612 21088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:13.724906 21088 log.cc:824] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:13.725663 21088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41: No bootstrap required, opened a new log
I20260619 16:58:13.726634 21088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } }
I20260619 16:58:13.726773 21088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 868543a1b73e43139eb12cb8b21c4c41, State: Initialized, Role: FOLLOWER
I20260619 16:58:13.726883 21088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } }
I20260619 16:58:13.727159 21258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } } }
I20260619 16:58:13.727231 21088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:13.727278 21258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:13.727286 21062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 16:58:13.727417 21062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:13.728168 21062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } }
I20260619 16:58:13.728479 21062 leader_election.cc:288] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [CANDIDATE]: Term 1 election: Requested vote from peers a40cd16f203d49b6bbb9cf671dbea6d9 (127.30.148.189:44219), 99948ac7e2234ea4a50bb80211a028d9 (127.30.148.188:36505)
I20260619 16:58:13.728762 21212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "868543a1b73e43139eb12cb8b21c4c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "99948ac7e2234ea4a50bb80211a028d9"
I20260619 16:58:13.728881 21212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:13.729807 21212 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 868543a1b73e43139eb12cb8b21c4c41 in term 1.
W20260619 16:58:13.730736 21275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:58:13.730860 21275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:58:13.730087 21037 leader_election.cc:302] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 868543a1b73e43139eb12cb8b21c4c41, 99948ac7e2234ea4a50bb80211a028d9; no voters:
I20260619 16:58:13.731184 21258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:13.731316 21258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [term 1 LEADER]: Becoming Leader. State: Replica: 868543a1b73e43139eb12cb8b21c4c41, State: Running, Role: LEADER
I20260619 16:58:13.731389 21258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } }
I20260619 16:58:13.731829 21258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 868543a1b73e43139eb12cb8b21c4c41. Latest consensus state: current_term: 1 leader_uuid: "868543a1b73e43139eb12cb8b21c4c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } } }
I20260619 16:58:13.731905 21258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:13.732539 21020 master_runner.cc:186] Error getting master registration for 127.30.148.189:44219: OK, instance_id { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" instance_seqno: 1781888293521453 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 16:58:13.734921 21020 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 16:58:13.735258 21280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:13.735862 21280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:13.738464 21212 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [term 1 FOLLOWER]: Refusing update from remote peer 868543a1b73e43139eb12cb8b21c4c41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:13.739079 21278 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:13.740628 21244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 868543a1b73e43139eb12cb8b21c4c41. Latest consensus state: current_term: 1 leader_uuid: "868543a1b73e43139eb12cb8b21c4c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } } }
I20260619 16:58:13.740746 21244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:13.741518 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.729938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56584 (local address 127.30.148.189:44219)
0619 16:58:13.730000 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.730002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.733847 (+ 3845us) server_negotiation.cc:407] Connection header received
0619 16:58:13.733944 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.733945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.733955 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.733984 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.734632 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.735343 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.736320 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.737047 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.737090 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.738547 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.738548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.738549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.738555 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.740182 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.740633 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.740635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.740635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.740692 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.740901 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.740902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.740902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.741038 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.741106 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.741333 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.741429 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:58:13.742754 21278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "868543a1b73e43139eb12cb8b21c4c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } } }
I20260619 16:58:13.742847 21278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:13.743033 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.729736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33666 (local address 127.30.148.190:36781)
0619 16:58:13.729776 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.729778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.730398 (+ 620us) server_negotiation.cc:407] Connection header received
0619 16:58:13.730495 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.730496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.730505 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.730534 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.731277 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.732036 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.733094 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.733417 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.733458 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.739917 (+ 6459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.739918 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.739919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.739925 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.741722 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.742144 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.742146 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.742146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.742216 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.742439 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.742440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.742440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.742573 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.742660 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.742883 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.742957 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:13.743285 21244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "868543a1b73e43139eb12cb8b21c4c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } } }
I20260619 16:58:13.743364 21244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:13.743637 21282 mvcc.cc:205] Tried to move back new op lower bound from 7298614451149860864 to 7298614451124518912. Current Snapshot: MvccSnapshot[applied={T|T < 7298614451149860864}]
I20260619 16:58:13.743873 21280 catalog_manager.cc:1347] Generated new cluster ID: 1d8a61b1694542c28b382e0f02b7aa3e
I20260619 16:58:13.743917 21280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:13.747486 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.730180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39472 (local address 127.30.148.188:36505)
0619 16:58:13.730253 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.730255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.740109 (+ 9854us) server_negotiation.cc:407] Connection header received
0619 16:58:13.740287 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.740288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.740298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.740328 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.741024 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.741704 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.742663 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.742966 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.743007 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.744359 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.744360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.744361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.744368 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.746069 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.746456 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.746457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.746457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.746512 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.746731 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.746732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.746732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.746853 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.746935 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.747329 (+ 394us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.747403 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 16:58:13.756055 21163 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } attempt: 3
Jun 19 16:58:13 dist-test-slave-xb9n krb5kdc[21007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.189@KRBTEST.COM for kudu/127.30.148.188@KRBTEST.COM
I20260619 16:58:13.765410 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.756287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36505 (local address 127.0.0.1:39474)
0619 16:58:13.756362 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:13.756372 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:13.756407 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:13.756614 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:13.756615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:13.756783 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:13.756823 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.756824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.758456 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.758457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.759266 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:13.759268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.759763 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.759764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:13.759818 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.760248 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:13.760252 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:13.761737 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:13.764351 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.764352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.764352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.764565 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.764780 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:13.764780 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:13.764781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:13.764836 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:13.765092 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:13.765093 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:13.765137 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 16:58:13.765197 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 16:58:13.765286 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 16:58:13.765746 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:13.756472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39474 (local address 127.30.148.188:36505)
0619 16:58:13.756504 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.756506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.756512 (+ 6us) server_negotiation.cc:407] Connection header received
0619 16:58:13.756529 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.756529 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.756537 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.756579 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.756899 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.757678 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.759364 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:13.759662 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:13.759716 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:13.761824 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:13.761825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:13.761825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:13.761836 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:13.763487 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.764647 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.764659 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.764660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.764713 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:13.764899 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:13.764899 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:13.764900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:13.765013 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:13.765068 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:13.765581 (+ 513us) server_negotiation.cc:299] Negotiation successful
0619 16:58:13.765662 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:58:13.767124 21163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9: Bootstrap starting.
I20260619 16:58:13.767848 21163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:13.768137 21163 log.cc:824] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:13.769716 21163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9: No bootstrap required, opened a new log
I20260619 16:58:13.771685 21163 raft_consensus.cc:348] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } }
I20260619 16:58:13.771823 21163 raft_consensus.cc:729] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a40cd16f203d49b6bbb9cf671dbea6d9, State: Initialized, Role: FOLLOWER
I20260619 16:58:13.771901 21163 consensus_queue.cc:260] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } }
I20260619 16:58:13.772217 21163 sys_catalog.cc:564] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:13.772267 21137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "868543a1b73e43139eb12cb8b21c4c41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9"
I20260619 16:58:13.772403 21137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:13.773197 21137 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 868543a1b73e43139eb12cb8b21c4c41 in term 1.
I20260619 16:58:13.773408 21136 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [term 1 FOLLOWER]: Refusing update from remote peer 868543a1b73e43139eb12cb8b21c4c41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:13.773645 21258 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:13.775461 21287 sys_catalog.cc:455] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "868543a1b73e43139eb12cb8b21c4c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } } }
I20260619 16:58:13.775563 21287 sys_catalog.cc:458] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:13.775619 21287 sys_catalog.cc:455] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 868543a1b73e43139eb12cb8b21c4c41. Latest consensus state: current_term: 1 leader_uuid: "868543a1b73e43139eb12cb8b21c4c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } } }
I20260619 16:58:13.775662 21287 sys_catalog.cc:458] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:13.775702 21287 sys_catalog.cc:455] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "868543a1b73e43139eb12cb8b21c4c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } } }
I20260619 16:58:13.775745 21287 sys_catalog.cc:458] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 16:58:13.782161 21302 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:58:13.782222 21302 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:58:13.783095 21295 mvcc.cc:205] Tried to move back new op lower bound from 7298614451149860864 to 7298614451124518912. Current Snapshot: MvccSnapshot[applied={T|T < 7298614451149860864}]
I20260619 16:58:13.787142 21258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "868543a1b73e43139eb12cb8b21c4c41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "868543a1b73e43139eb12cb8b21c4c41" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 36781 } } peers { permanent_uuid: "a40cd16f203d49b6bbb9cf671dbea6d9" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 44219 } } peers { permanent_uuid: "99948ac7e2234ea4a50bb80211a028d9" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36505 } } }
I20260619 16:58:13.787241 21258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:13.788128 21280 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:13.788851 21280 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:13.795048 21280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 868543a1b73e43139eb12cb8b21c4c41: Generated new TSK 0
I20260619 16:58:13.795259 21280 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 16:58:13.822335 21167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:13.820666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56598 (local address 127.30.148.189:44219)
0619 16:58:13.820711 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.820713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.821656 (+ 943us) server_negotiation.cc:407] Connection header received
0619 16:58:13.821768 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.821769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.821779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.821822 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.822268 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56598: BlockingRecv error: recv got EOF from 127.0.0.1:56598 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 16:58:13.823089 21090 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:13.820949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33672 (local address 127.30.148.190:36781)
0619 16:58:13.820994 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.820996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.822569 (+ 1573us) server_negotiation.cc:407] Connection header received
0619 16:58:13.822666 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.822667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.822677 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.822706 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.823028 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33672: BlockingRecv error: recv got EOF from 127.0.0.1:33672 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 16:58:13.823774 21242 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:13.821284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39482 (local address 127.30.148.188:36505)
0619 16:58:13.821321 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.821323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.823307 (+ 1984us) server_negotiation.cc:407] Connection header received
0619 16:58:13.823401 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.823402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.823412 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.823440 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.823721 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39482: BlockingRecv error: recv got EOF from 127.0.0.1:39482 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 16:58:13.826033 21167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:13.825221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.30.148.189:44219)
0619 16:58:13.825255 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.825256 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.825514 (+ 258us) server_negotiation.cc:407] Connection header received
0619 16:58:13.825620 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.825621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.825637 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.825666 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.825989 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56610: BlockingRecv error: recv got EOF from 127.0.0.1:56610 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 16:58:13.834069 21167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:13.829719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56622 (local address 127.30.148.189:44219)
0619 16:58:13.829751 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.829753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.833557 (+ 3804us) server_negotiation.cc:407] Connection header received
0619 16:58:13.833655 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.833656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.833666 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.833695 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.834025 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56622: BlockingRecv error: recv got EOF from 127.0.0.1:56622 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 16:58:13.837854 21167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:13.837111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56626 (local address 127.30.148.189:44219)
0619 16:58:13.837153 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.837154 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.837376 (+ 222us) server_negotiation.cc:407] Connection header received
0619 16:58:13.837470 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.837471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.837479 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.837506 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.837810 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56626: BlockingRecv error: recv got EOF from 127.0.0.1:56626 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260619 16:58:13.843716 21167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:13.842961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56638 (local address 127.30.148.189:44219)
0619 16:58:13.843000 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.843002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.843250 (+ 248us) server_negotiation.cc:407] Connection header received
0619 16:58:13.843343 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.843344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.843353 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.843380 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.843671 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56638: BlockingRecv error: recv got EOF from 127.0.0.1:56638 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 16:58:13.853734 21167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:13.852836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56642 (local address 127.30.148.189:44219)
0619 16:58:13.852870 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.852871 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.853099 (+ 228us) server_negotiation.cc:407] Connection header received
0619 16:58:13.853200 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.853201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.853211 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.853258 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.853679 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56642: BlockingRecv error: recv got EOF from 127.0.0.1:56642 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 16:58:13.871941 21167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:13.871069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56654 (local address 127.30.148.189:44219)
0619 16:58:13.871110 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.871112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.871352 (+ 240us) server_negotiation.cc:407] Connection header received
0619 16:58:13.871462 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.871463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.871473 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.871500 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.871885 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56654: BlockingRecv error: recv got EOF from 127.0.0.1:56654 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 16:58:13.906167 21167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:13.905341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56660 (local address 127.30.148.189:44219)
0619 16:58:13.905402 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.905404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.905519 (+ 115us) server_negotiation.cc:407] Connection header received
0619 16:58:13.905574 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.905575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.905586 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.905666 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.906098 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56660: BlockingRecv error: recv got EOF from 127.0.0.1:56660 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 16:58:13.972100 21167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:13.971318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56666 (local address 127.30.148.189:44219)
0619 16:58:13.971395 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:13.971397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:13.971483 (+ 86us) server_negotiation.cc:407] Connection header received
0619 16:58:13.971602 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:13.971603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:13.971616 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:13.971668 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:13.972035 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56666: BlockingRecv error: recv got EOF from 127.0.0.1:56666 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 16:58:14.102092 21167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:14.101248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56672 (local address 127.30.148.189:44219)
0619 16:58:14.101324 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:14.101327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:14.101444 (+ 117us) server_negotiation.cc:407] Connection header received
0619 16:58:14.101529 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:14.101530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:14.101541 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:14.101636 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:14.102004 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56672: BlockingRecv error: recv got EOF from 127.0.0.1:56672 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260619 16:58:14.359937 21167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:14.359373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56688 (local address 127.30.148.189:44219)
0619 16:58:14.359442 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:14.359443 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:14.359455 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:58:14.359532 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:14.359533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:14.359543 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:14.359587 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:14.359889 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56688: BlockingRecv error: recv got EOF from 127.0.0.1:56688 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 16:58:14.703773 21255 catalog_manager.cc:1259] Loaded cluster ID: 1d8a61b1694542c28b382e0f02b7aa3e
I20260619 16:58:14.703835 21255 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9: loading cluster ID for follower catalog manager: success
I20260619 16:58:14.704644 21255 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9: acquiring CA information for follower catalog manager: success
I20260619 16:58:14.704957 21255 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 99948ac7e2234ea4a50bb80211a028d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 16:58:14.782642 21302 catalog_manager.cc:1259] Loaded cluster ID: 1d8a61b1694542c28b382e0f02b7aa3e
I20260619 16:58:14.782720 21302 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9: loading cluster ID for follower catalog manager: success
I20260619 16:58:14.783567 21302 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9: acquiring CA information for follower catalog manager: success
I20260619 16:58:14.783886 21302 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a40cd16f203d49b6bbb9cf671dbea6d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 16:58:14.875800 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:14.873022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56698 (local address 127.30.148.189:44219)
0619 16:58:14.873166 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:14.873168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:14.873192 (+ 24us) server_negotiation.cc:407] Connection header received
0619 16:58:14.873255 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:14.873256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:14.873333 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:14.873438 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:14.873584 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:14.874160 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:14.874967 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:14.875192 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:14.875267 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:14.875451 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:14.875570 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:14.875664 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:14.875684 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 16:58:14.875704 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260619 16:58:14.879776 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:14.877574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39496 (local address 127.30.148.188:36505)
0619 16:58:14.877691 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:14.877693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:14.877701 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:14.877755 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:14.877756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:14.877787 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:14.877845 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:14.877936 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:14.878408 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:14.879073 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:14.879282 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:14.879335 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:14.879461 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:14.879545 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:14.879598 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:14.879636 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 16:58:14.879654 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260619 16:58:14.884027 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:14.881627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39512 (local address 127.30.148.188:36505)
0619 16:58:14.881666 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:14.881668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:14.881845 (+ 177us) server_negotiation.cc:407] Connection header received
0619 16:58:14.881940 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:14.881941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:14.881953 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:14.882014 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:14.882117 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:14.882539 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:14.883318 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:14.883552 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:14.883607 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:14.883785 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:14.883816 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:14.883871 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:14.883944 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 16:58:14.883964 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:58:14.884367 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:14.881406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56710 (local address 127.30.148.189:44219)
0619 16:58:14.881443 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:14.881445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:14.881680 (+ 235us) server_negotiation.cc:407] Connection header received
0619 16:58:14.881740 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:14.881741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:14.881753 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:14.881809 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:14.881966 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:14.882477 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:14.883550 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:14.883837 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:14.883913 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:14.884091 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:14.884121 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:14.884186 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:14.884262 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 16:58:14.884282 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:58:14.885226 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:14.881412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33674 (local address 127.30.148.190:36781)
0619 16:58:14.881783 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:14.881785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:14.881793 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:14.881831 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:14.881832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:14.881876 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:14.881980 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:14.882723 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:14.883475 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:14.884458 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:14.884804 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:14.884861 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:14.885002 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:14.885072 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:14.885122 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:14.885152 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 16:58:14.885171 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":43,"threads_started":1}
I20260619 16:58:14.890105 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 21020
I20260619 16:58:14.895499 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 21095
I20260619 16:58:14.900044 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 21170
2026-06-19T16:58:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2778 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:14 dist-test-slave-xb9n krb5kdc[21387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:14 dist-test-slave-xb9n krb5kdc[21387](info): set up 2 sockets
Jun 19 16:58:14 dist-test-slave-xb9n krb5kdc[21387](info): commencing operation
krb5kdc: starting...
W20260619 16:58:15.925611 31314 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/test-user.keytab.
Jun 19 16:58:15 dist-test-slave-xb9n krb5kdc[21387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.148.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
Entry for principal kudu/127.30.148.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab.
I20260619 16:58:16.036337 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:42825
--webserver_interface=127.30.148.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:35463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:42825,127.30.148.189:42851,127.30.148.188:36963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:16.115150 21400 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 16:58:16.115396 21400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:16.115428 21400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:16.115474 21400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:16.116577 21400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:16.116639 21400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:16.116664 21400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:16.116693 21400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:16.116724 21400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:16.118144 21400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:42825,127.30.148.189:42851,127.30.148.188:36963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.190.keytab
--principal=kudu/127.30.148.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.190:42825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.148.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:16.118404 21400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:16.118621 21400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:16.123342 21405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:16.123430 21400 server_base.cc:1034] running on GCE node
W20260619 16:58:16.123354 21408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:16.123373 21406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:16.123893 21400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:16.124172 21400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:16.125313 21400 hybrid_clock.cc:648] HybridClock initialized: now 1781888296125289 us; error 42 us; skew 500 ppm
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:16.127660 21400 init.cc:377] Logged in from keytab as kudu/127.30.148.190@KRBTEST.COM (short username kudu)
I20260619 16:58:16.128024 21400 webserver.cc:460] Webserver started at http://127.30.148.190:37291/ using document root <none> and password file <none>
I20260619 16:58:16.128206 21400 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:16.128249 21400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:16.128332 21400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:16.129288 21400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/data/instance:
uuid: "54d6ebdb97a94e4e9989305423a1cac2"
format_stamp: "Formatted at 2026-06-19 16:58:16 on dist-test-slave-xb9n"
server_key: "342a06551734ebfc723153197cd08eb0"
server_key_iv: "7901f0216cd51042b6dab132d459e37f"
server_key_version: "encryptionkey@0"
I20260619 16:58:16.129689 21400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/wal/instance:
uuid: "54d6ebdb97a94e4e9989305423a1cac2"
format_stamp: "Formatted at 2026-06-19 16:58:16 on dist-test-slave-xb9n"
server_key: "342a06551734ebfc723153197cd08eb0"
server_key_iv: "7901f0216cd51042b6dab132d459e37f"
server_key_version: "encryptionkey@0"
I20260619 16:58:16.130981 21400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:16.131596 21415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:16.131779 21400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:16.131867 21400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/wal
uuid: "54d6ebdb97a94e4e9989305423a1cac2"
format_stamp: "Formatted at 2026-06-19 16:58:16 on dist-test-slave-xb9n"
server_key: "342a06551734ebfc723153197cd08eb0"
server_key_iv: "7901f0216cd51042b6dab132d459e37f"
server_key_version: "encryptionkey@0"
I20260619 16:58:16.131938 21400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:16.141752 21400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:16.143357 21400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:16.143498 21400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:16.147212 21400 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.190:42825
I20260619 16:58:16.147244 21467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.190:42825 every 8 connection(s)
I20260619 16:58:16.147555 21400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:16.148339 21468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:16.149775 21468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:58:16.150341 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 21400
I20260619 16:58:16.150475 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-0/wal/instance
I20260619 16:58:16.150662 31314 external_mini_cluster.cc:1428] Setting key 1e002c7f3d1ec1d6581b793356faa49a
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.148.189@KRBTEST.COM; defaulting to no policy
I20260619 16:58:16.159056 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.150379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:42825 (local address 127.0.0.1:39234)
0619 16:58:16.150585 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.150620 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:16.151114 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:16.151395 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:16.151397 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:16.151691 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:16.151916 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.151921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.152885 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.152886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.153625 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.153627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.154115 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.154116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.154179 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.154662 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:16.154669 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:16.155926 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:16.157669 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.157674 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.157676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.157877 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.158112 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.158112 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.158113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.158197 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.158520 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:16.158524 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:16.158579 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 16:58:16.158673 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 16:58:16.158888 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":92,"thread_start_us":63,"threads_started":1}
I20260619 16:58:16.159062 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.150412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.30.148.190:42825)
0619 16:58:16.150581 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.150589 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.150766 (+ 177us) server_negotiation.cc:407] Connection header received
0619 16:58:16.151265 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.151267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.151286 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.151386 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.152021 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.152800 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.153720 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.154024 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.154141 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.156023 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.156030 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.156030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.156037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.157584 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.157966 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.157968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.157969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.158046 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.158263 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.158263 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.158264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.158450 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.158539 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.158704 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.158829 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260619 16:58:16.159868 21468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.189@KRBTEST.COM" created.
I20260619 16:58:16.160207 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.160096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:42851 (local address 127.0.0.1:45170)
0619 16:58:16.160142 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.160184 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:42851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 16:58:16.160311 21418 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.148.189:42851: connect: Connection refused (error 111)
I20260619 16:58:16.160737 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.160635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:42851 (local address 127.0.0.1:45180)
0619 16:58:16.160679 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.160709 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:42851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 16:58:16.160861 21468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.189:42851: Network error: Client connection negotiation failed: client connection to 127.30.148.189:42851: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.189.keytab.
Entry for principal kudu/127.30.148.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.189.keytab.
I20260619 16:58:16.173727 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.189:42851
--webserver_interface=127.30.148.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.189.keytab
--principal=kudu/127.30.148.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:35463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:42825,127.30.148.189:42851,127.30.148.188:36963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:58:16.210079 21468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } attempt: 1
I20260619 16:58:16.210510 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.210382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:42851 (local address 127.0.0.1:45194)
0619 16:58:16.210446 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.210482 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:42851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 16:58:16.210835 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.210742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:42851 (local address 127.0.0.1:45196)
0619 16:58:16.210785 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.210809 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:42851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 16:58:16.210919 21468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.189:42851: Network error: Client connection negotiation failed: client connection to 127.30.148.189:42851: connect: Connection refused (error 111)
W20260619 16:58:16.250547 21475 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 16:58:16.250728 21475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:16.250744 21475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:16.250789 21475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:16.251886 21475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:16.251924 21475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:16.251938 21475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:16.251984 21475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:16.252002 21475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:16.253427 21475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:42825,127.30.148.189:42851,127.30.148.188:36963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.189.keytab
--principal=kudu/127.30.148.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.189:42851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.148.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:16.253649 21475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:16.253854 21475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:16.258433 21481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:16.258495 21483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:16.258455 21475 server_base.cc:1034] running on GCE node
W20260619 16:58:16.258457 21480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:16.258805 21475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:16.259023 21475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:16.260156 21475 hybrid_clock.cc:648] HybridClock initialized: now 1781888296260107 us; error 66 us; skew 500 ppm
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:16.263134 21475 init.cc:377] Logged in from keytab as kudu/127.30.148.189@KRBTEST.COM (short username kudu)
I20260619 16:58:16.263598 21475 webserver.cc:460] Webserver started at http://127.30.148.189:46755/ using document root <none> and password file <none>
I20260619 16:58:16.263811 21475 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:16.263854 21475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:16.264015 21475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:16.265004 21475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/data/instance:
uuid: "2cb5e9aa8d20437389fbdd1d7c115be5"
format_stamp: "Formatted at 2026-06-19 16:58:16 on dist-test-slave-xb9n"
server_key: "eab348a192e2c5533e1ec7cb9010ad49"
server_key_iv: "4cde046ba36b4292cf2f9edc152c10d2"
server_key_version: "encryptionkey@0"
I20260619 16:58:16.265419 21475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/wal/instance:
uuid: "2cb5e9aa8d20437389fbdd1d7c115be5"
format_stamp: "Formatted at 2026-06-19 16:58:16 on dist-test-slave-xb9n"
server_key: "eab348a192e2c5533e1ec7cb9010ad49"
server_key_iv: "4cde046ba36b4292cf2f9edc152c10d2"
server_key_version: "encryptionkey@0"
I20260619 16:58:16.266686 21475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:16.267477 21490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:16.267690 21475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:16.267791 21475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/wal
uuid: "2cb5e9aa8d20437389fbdd1d7c115be5"
format_stamp: "Formatted at 2026-06-19 16:58:16 on dist-test-slave-xb9n"
server_key: "eab348a192e2c5533e1ec7cb9010ad49"
server_key_iv: "4cde046ba36b4292cf2f9edc152c10d2"
server_key_version: "encryptionkey@0"
I20260619 16:58:16.267874 21475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:16.279052 21468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } attempt: 2
I20260619 16:58:16.279428 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.279309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:42851 (local address 127.0.0.1:45200)
0619 16:58:16.279366 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.279402 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:42851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 16:58:16.279798 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.279706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:42851 (local address 127.0.0.1:45212)
0619 16:58:16.279761 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.279783 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.189:42851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 16:58:16.279904 21468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.189:42851: Network error: Client connection negotiation failed: client connection to 127.30.148.189:42851: connect: Connection refused (error 111)
I20260619 16:58:16.280194 21475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:16.281857 21475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:16.281957 21475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:16.285562 21475 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.189:42851
I20260619 16:58:16.285629 21542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.189:42851 every 8 connection(s)
I20260619 16:58:16.285917 21475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/data/info.pb
I20260619 16:58:16.286649 21543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:16.287957 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 21475
I20260619 16:58:16.288102 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-1/wal/instance
I20260619 16:58:16.288321 31314 external_mini_cluster.cc:1428] Setting key c099628bb8c8ef791434ede1ba3a8763
I20260619 16:58:16.288224 21543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.189@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.148.188@KRBTEST.COM; defaulting to no policy
I20260619 16:58:16.297629 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.288929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39242 (local address 127.30.148.190:42825)
0619 16:58:16.288993 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.288996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.289251 (+ 255us) server_negotiation.cc:407] Connection header received
0619 16:58:16.290001 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.290002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.290018 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.290088 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.290625 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.291116 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.292110 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.292452 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.292532 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.294871 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.294872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.294872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.294878 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.296354 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.296765 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.296765 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.296766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.296815 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.297021 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.297021 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.297022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.297116 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.297164 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.297511 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.297574 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 16:58:16.297776 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.288915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:42825 (local address 127.0.0.1:39242)
0619 16:58:16.289099 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.289159 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:16.289908 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:16.290143 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:16.290146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:16.290366 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:16.290540 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.290544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.291192 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.291193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.292024 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.292026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.292535 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.292536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.292650 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.293144 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:16.293152 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:16.294777 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:16.296427 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.296434 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.296437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.296700 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.296874 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.296875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.296875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.296968 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.297170 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:16.297174 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:16.297396 (+ 222us) client_negotiation.cc:769] Sending connection context
0619 16:58:16.297492 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 16:58:16.297596 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260619 16:58:16.298338 21543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.148.188@KRBTEST.COM" created.
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.189@KRBTEST.COM for kudu/127.30.148.189@KRBTEST.COM
I20260619 16:58:16.305634 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.298585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:42851 (local address 127.0.0.1:45226)
0619 16:58:16.298634 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.298645 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:16.298689 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:16.299014 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:16.299015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:16.299141 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:16.299180 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.299181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.300027 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.300028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.300507 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.300508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.300996 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.300997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.301057 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.301450 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:16.301455 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:16.302827 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:16.304483 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.304484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.304485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.304700 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.304895 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.304896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.304897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.304965 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.305236 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:16.305237 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:16.305296 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 16:58:16.305407 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 16:58:16.305505 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 16:58:16.305652 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.298680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45226 (local address 127.30.148.189:42851)
0619 16:58:16.298868 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.298873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.298893 (+ 20us) server_negotiation.cc:407] Connection header received
0619 16:58:16.298912 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.298914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.298925 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.299014 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.299266 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.299931 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.300601 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.300909 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.300996 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.302910 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.302917 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.302917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.302923 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.304395 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.304779 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.304782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.304782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.304829 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.305032 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.305033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.305033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.305165 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.305221 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.305458 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.305551 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":158,"thread_start_us":91,"threads_started":1}
I20260619 16:58:16.306242 21543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:58:16.306522 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.306435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37154)
0619 16:58:16.306463 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.306504 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260619 16:58:16.306667 21492 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
I20260619 16:58:16.307044 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.306960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37168)
0619 16:58:16.307006 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.307029 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 16:58:16.307173 21543 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:36963: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.148.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.188.keytab.
Entry for principal kudu/127.30.148.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.188.keytab.
I20260619 16:58:16.313573 31314 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLxMmiv/build/release/bin/kudu
/tmp/dist-test-taskLxMmiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.188:36963
--webserver_interface=127.30.148.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.188.keytab
--principal=kudu/127.30.148.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.148.148:35463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.148.190:42825,127.30.148.189:42851,127.30.148.188:36963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:58:16.356416 21543 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } attempt: 1
I20260619 16:58:16.356906 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.356756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37170)
0619 16:58:16.356826 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.356869 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 16:58:16.357291 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.357188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37184)
0619 16:58:16.357230 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.357256 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 16:58:16.357412 21543 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:36963: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
I20260619 16:58:16.371049 21468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } attempt: 3
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.189@KRBTEST.COM
I20260619 16:58:16.378796 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.371403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:42851 (local address 127.0.0.1:45240)
0619 16:58:16.371455 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.371465 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:16.371513 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:16.371692 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:16.371693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:16.371840 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:16.371901 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.371903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.372831 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.372831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.373341 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.373342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.373881 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.373882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.373944 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.374425 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:16.374430 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:16.375920 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:16.377888 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.377888 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.377889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.378055 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.378259 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.378260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.378260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.378310 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.378577 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:16.378578 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:16.378615 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 16:58:16.378661 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 16:58:16.378734 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 16:58:16.378908 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.371432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45240 (local address 127.30.148.189:42851)
0619 16:58:16.371492 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.371495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.371507 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:58:16.371593 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.371594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.371606 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.371674 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.371974 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.372753 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.373460 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.373806 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.373884 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.376011 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.376013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.376013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.376023 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.377798 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.378126 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.378128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.378128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.378191 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.378374 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.378375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.378376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.378510 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.378591 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.378707 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.378803 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 16:58:16.379431 21468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:58:16.379740 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.379648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37198)
0619 16:58:16.379690 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.379718 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 16:58:16.380038 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.379966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37212)
0619 16:58:16.380004 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.380025 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 16:58:16.380142 21468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:36963: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
W20260619 16:58:16.394335 21550 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 16:58:16.394523 21550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:16.394568 21550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:16.394608 21550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:16.395735 21550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:16.395773 21550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:16.395795 21550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:16.395831 21550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:16.395859 21550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:16.397311 21550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.148.148:35463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.148.190:42825,127.30.148.189:42851,127.30.148.188:36963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/krb5kdc/kudu_127.30.148.188.keytab
--principal=kudu/127.30.148.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.148.188:36963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.148.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:16.397615 21550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:16.397861 21550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:16.402199 21556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:16.402185 21555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:16.402186 21558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:16.402496 21550 server_base.cc:1034] running on GCE node
I20260619 16:58:16.402686 21550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:16.402897 21550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:16.404062 21550 hybrid_clock.cc:648] HybridClock initialized: now 1781888296404050 us; error 25 us; skew 500 ppm
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:16.406373 21550 init.cc:377] Logged in from keytab as kudu/127.30.148.188@KRBTEST.COM (short username kudu)
I20260619 16:58:16.406843 21550 webserver.cc:460] Webserver started at http://127.30.148.188:43311/ using document root <none> and password file <none>
I20260619 16:58:16.407075 21550 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:16.407116 21550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:16.407263 21550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:16.408254 21550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/data/instance:
uuid: "9dce0358e51043509fe105d462dc602d"
format_stamp: "Formatted at 2026-06-19 16:58:16 on dist-test-slave-xb9n"
server_key: "486de9ecbb3739288117b7767db73971"
server_key_iv: "d21a9e0864ad27efb76b013c3b94e657"
server_key_version: "encryptionkey@0"
I20260619 16:58:16.408594 21550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/wal/instance:
uuid: "9dce0358e51043509fe105d462dc602d"
format_stamp: "Formatted at 2026-06-19 16:58:16 on dist-test-slave-xb9n"
server_key: "486de9ecbb3739288117b7767db73971"
server_key_iv: "d21a9e0864ad27efb76b013c3b94e657"
server_key_version: "encryptionkey@0"
I20260619 16:58:16.409909 21550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:16.410565 21565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:16.410722 21550 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:16.410825 21550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/data,/tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/wal
uuid: "9dce0358e51043509fe105d462dc602d"
format_stamp: "Formatted at 2026-06-19 16:58:16 on dist-test-slave-xb9n"
server_key: "486de9ecbb3739288117b7767db73971"
server_key_iv: "d21a9e0864ad27efb76b013c3b94e657"
server_key_version: "encryptionkey@0"
I20260619 16:58:16.410900 21550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:16.411280 21468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } attempt: 1
I20260619 16:58:16.411607 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.411501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37228)
0619 16:58:16.411551 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.411578 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 16:58:16.411898 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.411815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37232)
0619 16:58:16.411855 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.411878 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 16:58:16.411990 21468 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:36963: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
I20260619 16:58:16.425575 21543 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } attempt: 2
I20260619 16:58:16.425976 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.425825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37244)
0619 16:58:16.425896 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.425927 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 16:58:16.426328 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.426225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37252)
0619 16:58:16.426272 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.426303 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 16:58:16.426430 21543 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.148.188:36963: Network error: Client connection negotiation failed: client connection to 127.30.148.188:36963: connect: Connection refused (error 111)
I20260619 16:58:16.427243 21550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:16.429431 21550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:16.429540 21550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:16.433010 21550 rpc_server.cc:307] RPC server started. Bound to: 127.30.148.188:36963
I20260619 16:58:16.433069 21617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.148.188:36963 every 8 connection(s)
I20260619 16:58:16.433452 21550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/data/info.pb
I20260619 16:58:16.434260 21618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:16.435796 21618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:58:16.437685 31314 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLxMmiv/build/release/bin/kudu as pid 21550
I20260619 16:58:16.437804 31314 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLxMmiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888036930978-31314-0/minicluster-data/master-2/wal/instance
I20260619 16:58:16.437988 31314 external_mini_cluster.cc:1428] Setting key 6247c3c6911d1302ab3d9d5c579d135b
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.188@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.188@KRBTEST.COM
I20260619 16:58:16.444870 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.436424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39244 (local address 127.30.148.190:42825)
0619 16:58:16.436496 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.436498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.436663 (+ 165us) server_negotiation.cc:407] Connection header received
0619 16:58:16.437094 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.437095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.437104 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.437164 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.437772 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.438529 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.439645 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.439822 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.439873 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.441848 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.441849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.441850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.441858 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.443379 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.443798 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.443799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.443799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.443857 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.444147 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.444148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.444148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.444315 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.444390 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.444681 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.444758 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 16:58:16.445037 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.436385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.190:42825 (local address 127.0.0.1:39244)
0619 16:58:16.436558 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.436589 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:16.437018 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:16.437211 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:16.437214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:16.437496 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:16.437682 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.437686 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.438607 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.438608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.439565 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.439567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.439894 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.439894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.440021 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.440474 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:16.440481 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:16.441777 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:16.443455 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.443462 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.443464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.443727 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.443933 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.443933 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.443934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.444065 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.444387 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:16.444393 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:16.444517 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 16:58:16.444631 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 16:58:16.444798 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260619 16:58:16.445672 21618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:58:16.446928 21622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.438363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37260 (local address 127.30.148.188:36963)
0619 16:58:16.438503 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.438507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.438559 (+ 52us) server_negotiation.cc:407] Connection header received
0619 16:58:16.438603 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.438606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.438629 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.438714 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.439274 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.440203 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.440986 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.441287 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.441368 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.443631 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.443638 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.443638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.443644 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.445623 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.446051 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.446056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.446056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.446109 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.446320 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.446321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.446321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.446501 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.446658 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.446754 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.446829 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":91,"threads_started":1}
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.188@KRBTEST.COM for kudu/127.30.148.189@KRBTEST.COM
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.190@KRBTEST.COM
I20260619 16:58:16.452840 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.445948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.189:42851 (local address 127.0.0.1:45248)
0619 16:58:16.446201 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.446211 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:16.446250 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:16.446395 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:16.446395 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:16.446496 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:16.446522 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.446523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.447384 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.447384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.448117 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.448120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.448497 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.448498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.448539 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.448932 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:16.448938 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:16.450286 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:16.451827 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.451828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.451829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.452024 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.452242 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.452243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.452243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.452301 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.452543 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:16.452544 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:16.452583 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 16:58:16.452670 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 16:58:16.452755 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239}
I20260619 16:58:16.452850 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.445999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.30.148.189:42851)
0619 16:58:16.446049 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.446051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.446257 (+ 206us) server_negotiation.cc:407] Connection header received
0619 16:58:16.446325 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.446326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.446336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.446387 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.446617 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.447308 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.448202 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.448394 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.448480 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.450372 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.450373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.450373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.450380 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.451717 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.452110 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.452111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.452112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.452168 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.452371 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.452372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.452372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.452477 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.452550 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.452695 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.452771 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:16.453200 21618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } has no permanent_uuid. Determining permanent_uuid...
I20260619 16:58:16.455368 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.447774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.30.148.190:42825)
0619 16:58:16.447829 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.447830 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.447839 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:16.447870 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.447871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.447886 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.447919 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.448444 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.448898 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.449757 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.450049 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.450114 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.452155 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.452157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.452157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.452164 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.453852 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.454626 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.454627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.454627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.454685 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.454886 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.454887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.454887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.455002 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.455170 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.455205 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.455282 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.188@KRBTEST.COM for kudu/127.30.148.188@KRBTEST.COM
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.148.189@KRBTEST.COM
I20260619 16:58:16.460124 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.453460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37272)
0619 16:58:16.453491 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.453502 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:16.453550 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:16.453664 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:16.453665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:16.453812 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:16.453846 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.453848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.454394 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.454395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.455067 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.455069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.455493 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.455494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.455542 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.455908 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:16.455913 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:16.457290 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:16.459074 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.459075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.459075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.459242 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.459465 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.459465 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.459466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.459534 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.459765 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:16.459765 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:16.459810 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 16:58:16.459893 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 16:58:16.459979 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 16:58:16.460186 21622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.453476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37272 (local address 127.30.148.188:36963)
0619 16:58:16.453509 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.453511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.453537 (+ 26us) server_negotiation.cc:407] Connection header received
0619 16:58:16.453604 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.453604 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.453612 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.453649 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.453909 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.454329 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.455145 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.455432 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.455495 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.457381 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.457382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.457383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.457389 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.458994 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.459313 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.459314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.459314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.459372 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.459607 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.459608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.459608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.459697 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.459746 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.459908 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.460018 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":19}
I20260619 16:58:16.461849 21618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d: Bootstrap starting.
I20260619 16:58:16.462630 21618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:16.462910 21618 log.cc:824] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:16.463071 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.455912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.30.148.189:42851)
0619 16:58:16.455952 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.455953 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.455994 (+ 41us) server_negotiation.cc:407] Connection header received
0619 16:58:16.456050 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.456051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.456060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.456123 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.456649 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.457117 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.457779 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.458009 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.458061 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.460536 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.460538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.460538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.460544 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.462006 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.462325 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.462326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.462326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.462376 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.462579 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.462580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.462580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.462685 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.462863 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.462902 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.462967 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:58:16.463977 21618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d: No bootstrap required, opened a new log
I20260619 16:58:16.464474 31314 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:16.465783 21618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } }
I20260619 16:58:16.465917 21618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dce0358e51043509fe105d462dc602d, State: Initialized, Role: FOLLOWER
I20260619 16:58:16.465996 21618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } }
I20260619 16:58:16.465519 21442 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "54d6ebdb97a94e4e9989305423a1cac2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39258
I20260619 16:58:16.466287 21618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:16.466252 21624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } } }
I20260619 16:58:16.466423 21624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [sys.catalog]: This master's current role is: FOLLOWER
W20260619 16:58:16.467923 21635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:58:16.467981 21635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:58:16.468344 21550 master_runner.cc:186] Error getting master registration for 127.30.148.190:42825: OK, instance_id { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" instance_seqno: 1781888296146140 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 16:58:16.471158 21550 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 16:58:16.487131 21468 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } attempt: 2
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.190@KRBTEST.COM for kudu/127.30.148.188@KRBTEST.COM
I20260619 16:58:16.494745 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.487394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37280)
0619 16:58:16.487460 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.487472 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:16.487511 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:16.487641 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:16.487641 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:16.487771 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:16.487810 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.487811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.488754 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.488755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.489584 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.489586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.490048 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.490049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.490104 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.490560 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:16.490564 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:16.492032 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:16.493640 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.493642 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.493642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.493871 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.494073 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.494074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.494074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.494137 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.494430 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:16.494431 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:16.494477 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 16:58:16.494550 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 16:58:16.494638 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 16:58:16.494750 21622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.487472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37280 (local address 127.30.148.188:36963)
0619 16:58:16.487524 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.487526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.487535 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:16.487565 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.487566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.487575 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.487639 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.487942 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.488675 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.489663 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.489968 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.490042 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.492106 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.492107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.492108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.492113 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.493561 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.493947 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.493948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.493949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.494002 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.494202 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.494202 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.494203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.494362 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.494426 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.494582 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.494658 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 16:58:16.496474 21468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2: Bootstrap starting.
I20260619 16:58:16.497166 21468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:16.497514 21468 log.cc:824] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:16.498340 21468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2: No bootstrap required, opened a new log
I20260619 16:58:16.499641 21468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } }
I20260619 16:58:16.499794 21468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d6ebdb97a94e4e9989305423a1cac2, State: Initialized, Role: FOLLOWER
I20260619 16:58:16.499876 21468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } }
I20260619 16:58:16.500191 21468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:16.500149 21638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } } }
I20260619 16:58:16.500268 21638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:16.500303 21442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 16:58:16.500394 21442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:16.501092 21442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } }
I20260619 16:58:16.501627 21592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54d6ebdb97a94e4e9989305423a1cac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9dce0358e51043509fe105d462dc602d"
I20260619 16:58:16.501765 21592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [term 0 FOLLOWER]: Advancing to term 1
W20260619 16:58:16.501997 21649 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:58:16.502049 21649 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:58:16.502635 21592 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54d6ebdb97a94e4e9989305423a1cac2 in term 1.
I20260619 16:58:16.502862 21442 leader_election.cc:288] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [CANDIDATE]: Term 1 election: Requested vote from peers 2cb5e9aa8d20437389fbdd1d7c115be5 (127.30.148.189:42851), 9dce0358e51043509fe105d462dc602d (127.30.148.188:36963)
I20260619 16:58:16.503219 21417 leader_election.cc:302] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54d6ebdb97a94e4e9989305423a1cac2, 9dce0358e51043509fe105d462dc602d; no voters:
I20260619 16:58:16.503326 21638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:16.503494 21638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [term 1 LEADER]: Becoming Leader. State: Replica: 54d6ebdb97a94e4e9989305423a1cac2, State: Running, Role: LEADER
I20260619 16:58:16.503576 21638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } }
I20260619 16:58:16.503937 21638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d6ebdb97a94e4e9989305423a1cac2. Latest consensus state: current_term: 1 leader_uuid: "54d6ebdb97a94e4e9989305423a1cac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } } }
I20260619 16:58:16.504004 21638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:16.504156 21400 master_runner.cc:186] Error getting master registration for 127.30.148.189:42851: OK, instance_id { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" instance_seqno: 1781888296284538 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 16:58:16.506726 21400 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 16:58:16.510257 21656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:16.510746 21656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:16.513386 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.504500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39272 (local address 127.30.148.190:42825)
0619 16:58:16.504536 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.504538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.504852 (+ 314us) server_negotiation.cc:407] Connection header received
0619 16:58:16.504947 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.504948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.504958 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.504985 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.505733 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.506412 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.507476 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.507770 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.507824 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.509354 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.509372 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.509373 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.509379 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.511600 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.512265 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.512266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.512267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.512325 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.512794 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.512795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.512795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.512917 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.512983 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.513180 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.513279 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:58:16.514976 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.506826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45260 (local address 127.30.148.189:42851)
0619 16:58:16.506870 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.506872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.506880 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:16.506902 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.506903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.506912 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.506981 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.507699 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.508410 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.509341 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.510113 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.510160 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.511580 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.511581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.511582 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.511588 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.513236 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.513917 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.513918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.513919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.513977 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.514187 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.514187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.514188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.514330 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.514420 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.514777 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.514854 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:58:16.515849 21592 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [term 1 FOLLOWER]: Refusing update from remote peer 54d6ebdb97a94e4e9989305423a1cac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:16.516232 21651 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:16.517423 21622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.505225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37282 (local address 127.30.148.188:36963)
0619 16:58:16.505264 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.505266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.505832 (+ 566us) server_negotiation.cc:407] Connection header received
0619 16:58:16.505976 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.505976 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.505998 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.506027 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.507184 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.507902 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.509291 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.509604 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.509642 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.514028 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.514029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.514029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.514036 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.515728 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.516275 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.516277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.516277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.516338 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.516715 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.516716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.516716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.516856 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.516920 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.517244 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.517318 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 16:58:16.517532 21543 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } attempt: 3
I20260619 16:58:16.517583 21624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d6ebdb97a94e4e9989305423a1cac2. Latest consensus state: current_term: 1 leader_uuid: "54d6ebdb97a94e4e9989305423a1cac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } } }
I20260619 16:58:16.517664 21624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:16.519483 21638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "54d6ebdb97a94e4e9989305423a1cac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } } }
I20260619 16:58:16.519572 21638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:16.519692 21624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "54d6ebdb97a94e4e9989305423a1cac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } } }
I20260619 16:58:16.519765 21624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:16.519672 21661 mvcc.cc:205] Tried to move back new op lower bound from 7298614462526619648 to 7298614462479224832. Current Snapshot: MvccSnapshot[applied={T|T < 7298614462526619648}]
I20260619 16:58:16.520022 21662 mvcc.cc:205] Tried to move back new op lower bound from 7298614462526619648 to 7298614462479224832. Current Snapshot: MvccSnapshot[applied={T|T < 7298614462526619648}]
I20260619 16:58:16.520686 21656 catalog_manager.cc:1347] Generated new cluster ID: 2a3737ad6a9f4518b7f93a4628200294
I20260619 16:58:16.520730 21656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:58:16 dist-test-slave-xb9n krb5kdc[21387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.148.189@KRBTEST.COM for kudu/127.30.148.188@KRBTEST.COM
I20260619 16:58:16.526427 21545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.517857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.148.188:36963 (local address 127.0.0.1:37296)
0619 16:58:16.517900 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:16.517909 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:16.517946 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:16.518102 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:16.518103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:16.518252 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:16.518292 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.518294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.519208 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.519209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.520057 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:16.520060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.520560 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.520561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:16.520632 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.521095 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:16.521100 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:16.522593 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:16.524984 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.524985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.524985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.525206 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.525634 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:16.525635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:16.525635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:16.525707 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:16.526137 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:16.526138 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:16.526192 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 16:58:16.526249 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 16:58:16.526335 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 16:58:16.526970 21622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:16.517904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37296 (local address 127.30.148.188:36963)
0619 16:58:16.517939 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.517941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.517949 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:16.518003 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.518004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.518019 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.518063 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.518396 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.519101 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.520160 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:16.520442 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:16.520500 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:16.522697 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:16.522698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:16.522699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:16.522705 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:16.524841 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.525306 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.525307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.525308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.525489 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:16.525800 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:16.525801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:16.525801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:16.526007 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:16.526591 (+ 584us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:16.526665 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 16:58:16.526801 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:58:16.528337 21543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5: Bootstrap starting.
I20260619 16:58:16.529009 21543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:16.529295 21543 log.cc:824] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:16.530055 21543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5: No bootstrap required, opened a new log
I20260619 16:58:16.531790 21543 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } }
I20260619 16:58:16.531910 21543 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb5e9aa8d20437389fbdd1d7c115be5, State: Initialized, Role: FOLLOWER
I20260619 16:58:16.531985 21543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } }
I20260619 16:58:16.532249 21543 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:16.532267 21517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54d6ebdb97a94e4e9989305423a1cac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5"
W20260619 16:58:16.532552 21496 tablet.cc:2369] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:58:16.533025 21517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:16.533859 21517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54d6ebdb97a94e4e9989305423a1cac2 in term 1.
I20260619 16:58:16.534127 21516 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [term 1 FOLLOWER]: Refusing update from remote peer 54d6ebdb97a94e4e9989305423a1cac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:16.534335 21668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } } }
I20260619 16:58:16.534449 21668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:16.534668 21665 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:16.536247 21682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d6ebdb97a94e4e9989305423a1cac2. Latest consensus state: current_term: 1 leader_uuid: "54d6ebdb97a94e4e9989305423a1cac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } } }
I20260619 16:58:16.536326 21682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:16.536353 21682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "54d6ebdb97a94e4e9989305423a1cac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } } }
I20260619 16:58:16.536382 21682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 16:58:16.537787 21679 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:58:16.537837 21679 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:58:16.541739 21680 mvcc.cc:205] Tried to move back new op lower bound from 7298614462526619648 to 7298614462479224832. Current Snapshot: MvccSnapshot[applied={T|T < 7298614462526619648}]
I20260619 16:58:16.542467 21651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "54d6ebdb97a94e4e9989305423a1cac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54d6ebdb97a94e4e9989305423a1cac2" member_type: VOTER last_known_addr { host: "127.30.148.190" port: 42825 } } peers { permanent_uuid: "2cb5e9aa8d20437389fbdd1d7c115be5" member_type: VOTER last_known_addr { host: "127.30.148.189" port: 42851 } } peers { permanent_uuid: "9dce0358e51043509fe105d462dc602d" member_type: VOTER last_known_addr { host: "127.30.148.188" port: 36963 } } }
I20260619 16:58:16.542568 21651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:16.542928 21656 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:16.543627 21656 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:16.551846 21656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54d6ebdb97a94e4e9989305423a1cac2: Generated new TSK 0
I20260619 16:58:16.552008 21656 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 16:58:16.577167 21471 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:16.575560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39276 (local address 127.30.148.190:42825)
0619 16:58:16.575603 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.575605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.576506 (+ 901us) server_negotiation.cc:407] Connection header received
0619 16:58:16.576612 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.576614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.576625 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.576656 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.577103 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39276: BlockingRecv error: recv got EOF from 127.0.0.1:39276 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 16:58:16.577946 21547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:16.575914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.30.148.189:42851)
0619 16:58:16.575950 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.575952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.577455 (+ 1503us) server_negotiation.cc:407] Connection header received
0619 16:58:16.577550 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.577551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.577562 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.577591 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.577894 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45268: BlockingRecv error: recv got EOF from 127.0.0.1:45268 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 16:58:16.578775 21622 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:16.576250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.30.148.188:36963)
0619 16:58:16.576287 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.576289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.578160 (+ 1871us) server_negotiation.cc:407] Connection header received
0619 16:58:16.578344 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.578345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.578356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.578385 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.578719 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37298: BlockingRecv error: recv got EOF from 127.0.0.1:37298 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 16:58:16.581173 21547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:16.580419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45284 (local address 127.30.148.189:42851)
0619 16:58:16.580452 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.580454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.580664 (+ 210us) server_negotiation.cc:407] Connection header received
0619 16:58:16.580769 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.580770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.580787 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.580815 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.581132 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45284: BlockingRecv error: recv got EOF from 127.0.0.1:45284 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 16:58:16.584079 21547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:16.583335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45300 (local address 127.30.148.189:42851)
0619 16:58:16.583367 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.583369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.583612 (+ 243us) server_negotiation.cc:407] Connection header received
0619 16:58:16.583706 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.583707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.583717 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.583746 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.584038 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45300: BlockingRecv error: recv got EOF from 127.0.0.1:45300 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 16:58:16.587803 21547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:16.587062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.30.148.189:42851)
0619 16:58:16.587093 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.587094 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.587342 (+ 248us) server_negotiation.cc:407] Connection header received
0619 16:58:16.587436 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.587437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.587446 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.587475 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.587767 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45306: BlockingRecv error: recv got EOF from 127.0.0.1:45306 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 16:58:16.593492 21547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:16.592729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45308 (local address 127.30.148.189:42851)
0619 16:58:16.592760 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.592762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.592982 (+ 220us) server_negotiation.cc:407] Connection header received
0619 16:58:16.593075 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.593076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.593085 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.593111 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.593455 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45308: BlockingRecv error: recv got EOF from 127.0.0.1:45308 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 16:58:16.606424 21547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:16.605640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45310 (local address 127.30.148.189:42851)
0619 16:58:16.605676 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.605677 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.605906 (+ 229us) server_negotiation.cc:407] Connection header received
0619 16:58:16.605996 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.605997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.606007 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.606035 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.606369 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45310: BlockingRecv error: recv got EOF from 127.0.0.1:45310 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 16:58:16.624498 21547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:16.623625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45312 (local address 127.30.148.189:42851)
0619 16:58:16.623664 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.623666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.623928 (+ 262us) server_negotiation.cc:407] Connection header received
0619 16:58:16.624031 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.624032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.624044 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.624073 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.624445 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45312: BlockingRecv error: recv got EOF from 127.0.0.1:45312 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 16:58:16.658493 21547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:16.657769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.30.148.189:42851)
0619 16:58:16.657815 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.657817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.657951 (+ 134us) server_negotiation.cc:407] Connection header received
0619 16:58:16.658024 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.658026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.658037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.658108 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.658424 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45314: BlockingRecv error: recv got EOF from 127.0.0.1:45314 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 16:58:16.724097 21547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:16.723495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45324 (local address 127.30.148.189:42851)
0619 16:58:16.723589 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.723591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.723601 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:16.723660 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.723661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.723669 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.723718 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.724033 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45324: BlockingRecv error: recv got EOF from 127.0.0.1:45324 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 16:58:16.853900 21547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:16.853227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45336 (local address 127.30.148.189:42851)
0619 16:58:16.853304 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:16.853306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:16.853388 (+ 82us) server_negotiation.cc:407] Connection header received
0619 16:58:16.853446 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:16.853446 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:16.853456 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:16.853509 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:16.853843 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45336: BlockingRecv error: recv got EOF from 127.0.0.1:45336 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260619 16:58:17.111847 21547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:17.111110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45344 (local address 127.30.148.189:42851)
0619 16:58:17.111175 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:17.111178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:17.111258 (+ 80us) server_negotiation.cc:407] Connection header received
0619 16:58:17.111344 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:17.111345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:17.111357 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:17.111433 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:17.111771 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45344: BlockingRecv error: recv got EOF from 127.0.0.1:45344 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 16:58:17.468606 21635 catalog_manager.cc:1259] Loaded cluster ID: 2a3737ad6a9f4518b7f93a4628200294
I20260619 16:58:17.468715 21635 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d: loading cluster ID for follower catalog manager: success
I20260619 16:58:17.469554 21635 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d: acquiring CA information for follower catalog manager: success
I20260619 16:58:17.469866 21635 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9dce0358e51043509fe105d462dc602d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 16:58:17.538234 21679 catalog_manager.cc:1259] Loaded cluster ID: 2a3737ad6a9f4518b7f93a4628200294
I20260619 16:58:17.538306 21679 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5: loading cluster ID for follower catalog manager: success
I20260619 16:58:17.539129 21679 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5: acquiring CA information for follower catalog manager: success
I20260619 16:58:17.539443 21679 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2cb5e9aa8d20437389fbdd1d7c115be5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 16:58:17.627442 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:17.624988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.30.148.189:42851)
0619 16:58:17.625098 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:17.625100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:17.625113 (+ 13us) server_negotiation.cc:407] Connection header received
0619 16:58:17.625191 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:17.625192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:17.625257 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:17.625332 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:17.625524 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:17.626019 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:17.626741 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:17.626931 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:17.626986 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:17.627114 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:17.627220 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:17.627279 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:17.627310 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 16:58:17.627326 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260619 16:58:17.631369 21756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:17.629218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.30.148.188:36963)
0619 16:58:17.629322 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:17.629324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:17.629333 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:17.629408 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:17.629409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:17.629454 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:17.629528 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:17.629612 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:17.630057 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:17.630725 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:17.630919 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:17.630982 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:17.631096 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:17.631204 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:17.631260 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:17.631294 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 16:58:17.631310 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":62,"threads_started":1}
I20260619 16:58:17.636081 21756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:17.633194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37330 (local address 127.30.148.188:36963)
0619 16:58:17.633231 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:17.633233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:17.633240 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:58:17.633260 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:17.633261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:17.633272 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:17.633326 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:17.633457 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:17.634173 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:17.635300 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:17.635599 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:17.635645 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:17.635834 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:17.635865 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:17.635892 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:17.635975 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 16:58:17.635994 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:17.636194 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:17.633213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45350 (local address 127.30.148.189:42851)
0619 16:58:17.633258 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:17.633259 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:17.633465 (+ 206us) server_negotiation.cc:407] Connection header received
0619 16:58:17.633583 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:17.633584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:17.633596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:17.633651 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:17.633805 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:17.634356 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:17.635351 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:17.635647 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:17.635710 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:17.635922 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:17.635949 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:17.635994 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:17.636094 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 16:58:17.636112 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 16:58:17.636389 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:17.633200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39286 (local address 127.30.148.190:42825)
0619 16:58:17.633416 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:17.633417 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:17.633424 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:58:17.633466 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:17.633467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:17.633549 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:17.633633 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:17.633763 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:17.634488 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:17.635308 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:17.635601 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:17.635665 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:17.635839 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:17.635935 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:17.635996 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:17.636296 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 16:58:17.636317 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":58,"threads_started":1}
I20260619 16:58:17.638094 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 21400
I20260619 16:58:17.643317 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 21475
I20260619 16:58:17.648116 31314 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLxMmiv/build/release/bin/kudu with pid 21550
2026-06-19T16:58:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2748 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5526 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (260725 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST